Sep 08 20:25:13.725275 ip-10-0-132-228 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 20:25:14.159489 ip-10-0-132-228 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 20:25:14.159489 ip-10-0-132-228 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 20:25:14.159489 ip-10-0-132-228 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 20:25:14.159489 ip-10-0-132-228 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 20:25:14.159489 ip-10-0-132-228 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 20:25:14.160518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.160369    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 20:25:14.166348 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166326    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:14.166348 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166345    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:14.166348 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166349    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:14.166348 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166352    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166355    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166359    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166361    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166364    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166367    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166370    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166374    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166377    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166379    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166382    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166385    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166388    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166396    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166399    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166402    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166405    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166408    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166411    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166413    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:14.166486 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166416    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166418    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166421    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166424    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166427    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166429    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166432    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166435    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166437    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166440    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166443    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166445    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166448    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166450    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166455    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166460    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166464    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166467    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166470    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:14.166984 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166473    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166476    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166478    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166481    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166484    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166488    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166491    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166494    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166496    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166499    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166502    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166504    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166507    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166510    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166512    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166515    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166519    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166522    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166527    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166530    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:14.167449 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166533    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166537    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166541    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166544    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166547    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166549    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166552    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166556    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166558    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166561    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166564    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166566    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166569    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166571    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166574    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166577    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166579    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166583    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166586    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166588    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:14.167956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166591    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166593    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166596    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166598    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166601    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166603    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166606    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166609    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166612    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166615    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166618    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166620    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166623    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166625    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166628    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166631    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.166633    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167379    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167387    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167390    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:14.168440 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167393    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167396    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167399    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167402    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167407    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167411    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167414    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167417    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167419    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167422    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167425    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167428    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167431    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167433    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167436    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167438    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167441    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167443    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167446    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:14.168942 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167449    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167452    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167455    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167458    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167461    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167464    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167467    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167470    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167473    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167475    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167478    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167480    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167483    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167485    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167488    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167490    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167493    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167495    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167498    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167501    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:14.169420 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167503    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167506    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167510    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167513    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167517    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167520    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167522    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167525    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167527    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167530    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167532    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167535    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167538    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167540    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167543    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167546    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167548    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167551    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167553    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:14.169930 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167555    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167558    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167561    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167563    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167566    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167568    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167571    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167575    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167579    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167582    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167584    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167587    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167589    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167592    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167594    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167597    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167600    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167602    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167605    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167608    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:14.170405 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167610    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167613    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167615    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167618    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167621    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167623    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167626    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167629    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167633    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167635    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167638    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167640    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167643    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167645    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167648    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167650    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167653    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.167655    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168084    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168090    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:14.170907 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168094    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168097    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168100    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168103    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168105    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168108    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168111    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168114    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168118    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168122    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168125    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168129    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168132    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168135    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168138    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168141    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168144    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168147    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168149    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168152    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:14.171416 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168155    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168157    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168160    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168162    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168165    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168168    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168171    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168174    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168176    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168179    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168181    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168184    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168186    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168189    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168191    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168194    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168196    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168199    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168201    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168204    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:14.171927 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168206    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168209    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168212    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168218    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168221    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168224    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168226    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168229    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168231    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168234    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168237    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168239    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168241    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168244    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168246    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168249    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168252    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168254    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168257    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168261    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:14.172407 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168264    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168267    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168269    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168272    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168275    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168277    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168280    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168283    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168285    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168287    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168290    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168292    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168295    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168297    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168300    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168305    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168309    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168312    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168314    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:14.172932 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168317    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168320    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168322    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168325    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168327    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168330    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168332    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168335    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168337    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168340    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168343    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168347    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168350    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168352    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168355    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168357    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168360    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.168362    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168449    2561 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168484    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 20:25:14.173412 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168491    2561 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168495    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168499    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168503    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168507    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168511    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168514    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168519    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168524    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168527    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168531    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168534    2561 flags.go:64] FLAG: --cgroup-root=""
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168537    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168540    2561 flags.go:64] FLAG: --client-ca-file=""
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168543    2561 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168546    2561 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168550    2561 flags.go:64] FLAG: --cluster-domain=""
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168553    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168556    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168559    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168562    2561 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168566    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168569    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 20:25:14.174050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168572    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168576    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168579    2561 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168582    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168585    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168588    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168591    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168595    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168598    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168601    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168603    2561 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168606    2561 flags.go:64] FLAG: --enable-server="true"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168609    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168614    2561 flags.go:64] FLAG: --event-burst="100"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168617    2561 flags.go:64] FLAG: --event-qps="50"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168620    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168624    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168630    2561 flags.go:64] FLAG: --eviction-hard=""
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168634    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168637    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168642    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168645    2561 flags.go:64] FLAG: --eviction-soft=""
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168649    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168651    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168654    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 20:25:14.174634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168657    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168660    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168663    2561 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168666    2561 flags.go:64] FLAG: --feature-gates=""
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168670    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168673    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168676    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168679    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168682    2561 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168685    2561 flags.go:64] FLAG: --help="false"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168688    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168691    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168694    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168697    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168701    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168704    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168706    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168710    2561 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168712    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168715    2561 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168718    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168721    2561 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168724    2561 flags.go:64] FLAG: --kube-reserved=""
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168728    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 20:25:14.175274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168732    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168735    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168737    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168740    2561 flags.go:64] FLAG: --lock-file=""
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168744    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168747    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168750    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168756    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168759    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168762    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168765    2561 flags.go:64] FLAG: --logging-format="text"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168768    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168771    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168774    2561 flags.go:64] FLAG: --manifest-url=""
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168777    2561 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168781    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168785    2561 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168789    2561 flags.go:64] FLAG: --max-pods="110"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168792    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168795    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168798    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168801    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168804    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168807    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168809    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 20:25:14.175850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168817    2561 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168820    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168823    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168826    2561 flags.go:64] FLAG: --pod-cidr=""
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168829    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168835    2561 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168839    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168842    2561 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168846    2561 flags.go:64] FLAG: --port="10250"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168849    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168852    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ce702d9fc3200aae"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168855    2561 flags.go:64] FLAG: --qos-reserved=""
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168859    2561 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168862    2561 flags.go:64] FLAG: --register-node="true"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168865    2561 flags.go:64] FLAG: --register-with-taints=""
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168869    2561 flags.go:64] FLAG: --registry-burst="10"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168872    2561 flags.go:64] FLAG: --registry-qps="5"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168874    2561 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168877    2561 flags.go:64] FLAG: --reserved-memory=""
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168881    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168884    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168887    2561 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168890    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168906    2561 flags.go:64] FLAG: --runonce="false"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168909    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 20:25:14.176527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168912    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168915    2561 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168918    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168921    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168925    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168928    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168931    2561 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168934    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168937    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168940    2561 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168943    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168946    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168949    2561 flags.go:64] FLAG: --system-cgroups=""
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168954    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168960    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168963    2561 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168966    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168971    2561 flags.go:64] FLAG: --tls-min-version=""
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168973    2561 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168976    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168980    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168983    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168985    2561 flags.go:64] FLAG: --v="2"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168989    2561 flags.go:64] FLAG: --version="false"
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168993    2561 flags.go:64] FLAG: --vmodule=""
Sep 08 20:25:14.177157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.168997    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.169000    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169126    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169130    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169133    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169136    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169139    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169142    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169145    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169147    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169150    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169152    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169156    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169158    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169161    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169165    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169169    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169172    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169175    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:14.177802 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169177    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169182    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169185    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169188    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169190    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169193    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169195    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169198    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169200    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169204    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169206    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169209    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169211    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169214    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169217    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169220    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169222    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169225    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169227    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169230    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:14.178284 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169232    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169235    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169238    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169240    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169243    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169245    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169247    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169250    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169252    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169255    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169258    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169260    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169262    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169267    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169269    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169272    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169275    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169277    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169280    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169282    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:14.178759 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169285    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169287    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169291    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169293    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169296    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169299    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169301    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169304    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169306    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169309    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169311    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169314    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169316    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169319    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169321    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169324    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169326    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169329    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169331    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169334    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:14.179270 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169336    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169339    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169341    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169345    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169349    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169353    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169356    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169359    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169362    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169364    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169367    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169369    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169372    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169375    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169378    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169382    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169384    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169387    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169389    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:14.179753 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169392    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169394    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.169397    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.170094    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.178313    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.178329    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.178346    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.178352    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178422    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178427    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178431    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178433    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178436    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178440    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178442    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:14.180243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178445    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178448    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178451    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178454    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178456    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178459    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178462    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178465    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178468    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178470    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178474    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178478    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178481    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178484    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178487    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178489    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178492    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178494    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178497    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178500    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:14.180619 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178502    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178505    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178509    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178512    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178515    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178518    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178522    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178525    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178528    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178531    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178535    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178538    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178541    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178545    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178548    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178551    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178554    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178556    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178559    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:14.181113 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178562    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178564    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178567    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178569    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178572    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178575    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178577    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178579    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178582    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178584    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178587    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178590    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178592    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178595    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178597    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178600    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178604    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178606    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178609    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178611    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:14.181584 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178614    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178616    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178619    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178621    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178624    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178627    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178629    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178632    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178635    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178637    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178640    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178642    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178645    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178648    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178650    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178653    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178655    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178658    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178660    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:14.182168 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178663    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178665    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178668    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178671    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178674    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178676    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178680    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178682    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178685    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178687    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178690    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178693    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178696    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178698    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.178703    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 20:25:14.182626 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178849    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178855    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178858    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178861    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178864    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178867    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178869    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178872    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178875    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178878    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178881    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178884    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178887    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178890    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178907    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178912    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178915    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178917    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178921    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178924    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:14.183018 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178927    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178929    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178932    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178935    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178937    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178940    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178943    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178945    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178949    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178951    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178954    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178966    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178969    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178972    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178974    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178977    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178979    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178982    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178984    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178987    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:14.183498 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178990    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178992    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178995    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.178998    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179000    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179003    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179006    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179008    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179010    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179013    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179015    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179018    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179021    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179024    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179026    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179029    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179031    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179034    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179036    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:14.184016 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179039    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179041    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179045    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179048    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179050    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179052    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179055    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179057    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179060    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179062    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179065    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179068    2561 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179070    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179072    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179076    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179080    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179082    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179085    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179088    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179090    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:14.184512 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179094    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179097    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179100    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179103    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179105    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179108    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179111    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179113    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179116    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179118    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179120    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179124    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179127    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179130    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179133    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179136    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179139    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179142    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179145    2561 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:14.185006 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:14.179147    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:14.185463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.179152    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 20:25:14.185463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.179286    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 20:25:14.185463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.183927    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 20:25:14.185463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.185033    2561 server.go:1019] "Starting client certificate rotation"
Sep 08 20:25:14.185463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.185133    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 20:25:14.185959 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.185946    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 20:25:14.188039 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.188009    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 20:25:14.189518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.189501    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 20:25:14.198758 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.198736    2561 log.go:25] "Validated CRI v1 runtime API"
Sep 08 20:25:14.202851 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.202831    2561 log.go:25] "Validated CRI v1 image API"
Sep 08 20:25:14.203744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.203724    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 20:25:14.207073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.207052    2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a4c06563-d440-46a3-87f7-f18b86463d3b:/dev/nvme0n1p3 ce13e7c7-237d-45d6-8f07-22546ebdd29e:/dev/nvme0n1p4]
Sep 08 20:25:14.207164 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.207072    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 20:25:14.213430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.213288    2561 manager.go:216] Machine: {Timestamp:2026-09-08 20:25:14.211807263 +0000 UTC m=+0.376977834 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100975 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec292aa551895f26f593bf56f000274d SystemUUID:ec292aa5-5189-5f26-f593-bf56f000274d BootID:08a8f17f-c0f1-420c-8f51-1d07552c012c Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5d:ca:c4:02:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5d:ca:c4:02:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:56:c1:e2:9e:a9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 20:25:14.213430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.213410    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 20:25:14.213588 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.213501    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 20:25:14.217037 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.217000    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 20:25:14.217249 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.217043    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-228.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 20:25:14.218039 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.218025    2561 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 20:25:14.218078 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.218046    2561 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 20:25:14.218078 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.218057    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 20:25:14.219682 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.219666    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 20:25:14.220519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.220493    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 20:25:14.221517 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.221502    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 20:25:14.226360 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.226342    2561 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 20:25:14.226530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.226519    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 20:25:14.226602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.226558    2561 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 20:25:14.226602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.226571    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 20:25:14.226602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.226593    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 20:25:14.226710 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.226606    2561 kubelet.go:398] "Adding apiserver pod source"
Sep 08 20:25:14.226710 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.226618    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 20:25:14.227093 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.227080    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 20:25:14.227138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.227104    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 20:25:14.229800 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.229772    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 20:25:14.233585 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.233569    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 20:25:14.233634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.233602    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 20:25:14.233818 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.233804    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 20:25:14.233855 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.233831    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 20:25:14.233855 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.233844    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 20:25:14.233938 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.233855    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 20:25:14.233938 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.233871    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 20:25:14.233938 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.233882    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 20:25:14.234477 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.234466    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 20:25:14.234477 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.234476    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 20:25:14.234554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.234485    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 20:25:14.234554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.234491    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 20:25:14.234554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.234500    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 20:25:14.234554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.234508    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 20:25:14.235347 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.235336    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 20:25:14.235347 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.235347    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 20:25:14.239318 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.239300    2561 server.go:1268] "Started kubelet"
Sep 08 20:25:14.239439 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.239358    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 20:25:14.239632 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.239574    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 20:25:14.240237 ip-10-0-132-228 systemd[1]: Started Kubernetes Kubelet.
Sep 08 20:25:14.240457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.240241    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 20:25:14.240763 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.240651    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-228.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 20:25:14.240913 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.240810    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 20:25:14.240913 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.240835    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-228.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 20:25:14.245181 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.245157    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 20:25:14.246202 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.246178    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 20:25:14.246408 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.246392    2561 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 20:25:14.249128 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.248001    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-228.ec2.internal.18d37244333e16f2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-228.ec2.internal,UID:ip-10-0-132-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-228.ec2.internal,},FirstTimestamp:2026-09-08 20:25:14.239268594 +0000 UTC m=+0.404439223,LastTimestamp:2026-09-08 20:25:14.239268594 +0000 UTC m=+0.404439223,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-228.ec2.internal,}"
Sep 08 20:25:14.250394 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.250377    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 20:25:14.250458 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.250405    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 20:25:14.250558 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.250534    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:14.251226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251195    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 20:25:14.251354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251334    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 20:25:14.251416 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251358    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 20:25:14.251472 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251448    2561 factory.go:55] Registering systemd factory
Sep 08 20:25:14.251532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251475    2561 factory.go:223] Registration of the systemd container factory successfully
Sep 08 20:25:14.251532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251505    2561 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 20:25:14.251532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251513    2561 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 20:25:14.251849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251829    2561 factory.go:153] Registering CRI-O factory
Sep 08 20:25:14.251849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251850    2561 factory.go:223] Registration of the crio container factory successfully
Sep 08 20:25:14.251997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251925    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 20:25:14.251997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251945    2561 factory.go:103] Registering Raw factory
Sep 08 20:25:14.251997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.251972    2561 manager.go:1196] Started watching for new ooms in manager
Sep 08 20:25:14.252384 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.252372    2561 manager.go:349] Starting recovery of all containers
Sep 08 20:25:14.261569 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.261419    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-228.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 08 20:25:14.261569 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.261514    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 08 20:25:14.262506 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.262491    2561 manager.go:354] Recovery completed
Sep 08 20:25:14.267032 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.267017    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:14.268555 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.268530    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j5226"
Sep 08 20:25:14.269523 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.269434    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:14.269523 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.269461    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:14.269523 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.269475    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:14.270126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.270113    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 20:25:14.270211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.270196    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 20:25:14.270262 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.270221    2561 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 20:25:14.270262 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.270135    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:14.271068 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.271050    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:14.271155 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.271084    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:14.271155 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.271099    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:14.272528 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.272512    2561 policy_none.go:49] "None policy: Start"
Sep 08 20:25:14.272528 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.272530    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 20:25:14.272628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.272541    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 20:25:14.274329 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.274313    2561 policy_none.go:47] "Start"
Sep 08 20:25:14.277759 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.277742    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j5226"
Sep 08 20:25:14.280851 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.280781    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-228.ec2.internal.18d37244350a8d2d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-228.ec2.internal,UID:ip-10-0-132-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-228.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-228.ec2.internal,},FirstTimestamp:2026-09-08 20:25:14.269445421 +0000 UTC m=+0.434615988,LastTimestamp:2026-09-08 20:25:14.269445421 +0000 UTC m=+0.434615988,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-228.ec2.internal,}"
Sep 08 20:25:14.306498 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.306479    2561 manager.go:337] "Starting Device Plugin manager"
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.306518    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.306528    2561 server.go:87] "Starting device plugin registration server"
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.306825    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.306838    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.306946    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.307044    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.307241    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.307577    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 20:25:14.330775 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.307627    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:14.355316 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.355277    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 20:25:14.356594 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.356568    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 20:25:14.356720 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.356628    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 20:25:14.356720 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.356660    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 20:25:14.356817 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.356764    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 20:25:14.359112 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.359092    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:14.406955 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.406919    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:14.408136 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.408115    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:14.408252 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.408151    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:14.408252 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.408165    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:14.408252 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.408200    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.416603 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.416551    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.416603 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.416576    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-228.ec2.internal\": node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:14.449171 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.449142    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:14.457380 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.457352    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal"]
Sep 08 20:25:14.457452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.457427    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:14.459560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.459543    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:14.459642 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.459576    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:14.459642 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.459586    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:14.462645 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.462634    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:14.462780 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.462766    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.462821 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.462798    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:14.464126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.464106    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:14.464226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.464132    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:14.464226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.464146    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:14.464975 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.464955    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:14.465063 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.464988    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:14.465063 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.464998    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:14.466848 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.466833    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.466946 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.466859    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:14.468115 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.468100    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:14.468196 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.468122    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:14.468196 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.468132    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:14.494779 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.494747    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-228.ec2.internal\" not found" node="ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.499195 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.499175    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-228.ec2.internal\" not found" node="ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.549484 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.549445    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:14.552749 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.552726    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf3209fa1c2452dc88935573fa8e83a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal\" (UID: \"bf3209fa1c2452dc88935573fa8e83a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.552848 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.552760    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf3209fa1c2452dc88935573fa8e83a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal\" (UID: \"bf3209fa1c2452dc88935573fa8e83a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.552848 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.552784    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd93909c83a29abfbb61422c635288b0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-228.ec2.internal\" (UID: \"dd93909c83a29abfbb61422c635288b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.652342 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.652303    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:14.653582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.653556    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf3209fa1c2452dc88935573fa8e83a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal\" (UID: \"bf3209fa1c2452dc88935573fa8e83a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.653658 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.653595    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd93909c83a29abfbb61422c635288b0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-228.ec2.internal\" (UID: \"dd93909c83a29abfbb61422c635288b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.653658 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.653615    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf3209fa1c2452dc88935573fa8e83a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal\" (UID: \"bf3209fa1c2452dc88935573fa8e83a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.653746 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.653661    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf3209fa1c2452dc88935573fa8e83a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal\" (UID: \"bf3209fa1c2452dc88935573fa8e83a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.653746 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.653660    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf3209fa1c2452dc88935573fa8e83a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal\" (UID: \"bf3209fa1c2452dc88935573fa8e83a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.653746 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.653708    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd93909c83a29abfbb61422c635288b0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-228.ec2.internal\" (UID: \"dd93909c83a29abfbb61422c635288b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.753074 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.752958    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:14.797354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.797314    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.802432 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:14.802193    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:14.853622 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.853579    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:14.954343 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:14.954304    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:15.055222 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.055145    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:15.155592 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.155561    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-228.ec2.internal\" not found"
Sep 08 20:25:15.166865 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.166845    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:15.185059 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.185031    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 20:25:15.185216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.185194    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 20:25:15.185255 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.185224    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 20:25:15.220137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.220109    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:15.227670 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.227649    2561 apiserver.go:52] "Watching apiserver"
Sep 08 20:25:15.237145 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.237124    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 20:25:15.238297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.238275    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-rqs79","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq","openshift-cluster-node-tuning-operator/tuned-5b5hz","openshift-dns/node-resolver-psrxw","openshift-image-registry/node-ca-sktlj","openshift-multus/multus-gtfhv","openshift-multus/network-metrics-daemon-qxhpg","openshift-network-operator/iptables-alerter-2v5m2","kube-system/konnectivity-agent-vnw88","openshift-multus/multus-additional-cni-plugins-4vfsr","openshift-network-diagnostics/network-check-target-kslns","openshift-ovn-kubernetes/ovnkube-node-5bgdt"]
Sep 08 20:25:15.240813 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.240787    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.242965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.242940    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.243295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.243276    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 20:25:15.243588 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.243571    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 20:25:15.243653 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.243622    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5xwfw\""
Sep 08 20:25:15.243653 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.243640    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 20:25:15.245090 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.245066    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.245611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.245592    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j5d7h\""
Sep 08 20:25:15.245961 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.245942    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:25:15.246053 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.246008    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 20:25:15.248303 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.247797    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 20:25:15.248303 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.247921    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 20:25:15.248469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.248387    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:15.248521 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.248470    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-59rt8\""
Sep 08 20:25:15.248634 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.248599    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:15.248701 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.248467    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 20:25:15.250693 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.250672    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 20:25:15.250799 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.250699    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:15.250799 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.250675    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.253075 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.253058    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:15.253194 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.253107    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:15.253296 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.253275    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 20:25:15.253352 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.253296    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 20:25:15.253352 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.253301    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8nls4\""
Sep 08 20:25:15.253352 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.253307    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 20:25:15.253511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.253388    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 20:25:15.253623 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.253607    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 20:25:15.255428 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.255409    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.256848 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.256814    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.256940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.256854    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-registration-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.256940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.256913    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.257026 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.256950    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8crm8\" (UniqueName: \"kubernetes.io/projected/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-kube-api-access-8crm8\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.257026 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.256993    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-socket-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.257026 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-sys-fs\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.257026 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257024    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-host\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257046    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ebae549-d876-40d4-a52c-025f6b21dde0-serviceca\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.257168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257071    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-system-cni-dir\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.257168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257107    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cni-binary-copy\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.257168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257134    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:15.257168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257152    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g77h\" (UniqueName: \"kubernetes.io/projected/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-kube-api-access-9g77h\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257186    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-kubernetes\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-sys\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257245    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-lib-modules\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257263    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-var-lib-kubelet\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257286    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-tuned\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257309    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8l62\" (UniqueName: \"kubernetes.io/projected/f559af35-ddd5-4034-93fb-94b0e072bbd0-kube-api-access-q8l62\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257328    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-systemd\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257342    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-run\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ebae549-d876-40d4-a52c-025f6b21dde0-host\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257381    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn6wm\" (UniqueName: \"kubernetes.io/projected/2ebae549-d876-40d4-a52c-025f6b21dde0-kube-api-access-bn6wm\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257404    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257424    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-device-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257447    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqhtm\" (UniqueName: \"kubernetes.io/projected/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-kube-api-access-pqhtm\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257464    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-modprobe-d\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257483    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-tmp\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257509    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cnibin\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257536    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysconfig\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257554    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysctl-d\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257597    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257636    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-etc-selinux\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257686    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysctl-conf\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.257716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257714    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:15.258167 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257772    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 20:25:15.258167 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.257838    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.258167 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.257963    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:15.258440 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.258424    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 20:25:15.258547 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.258522    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 20:25:15.258630 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.258547    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 20:25:15.258630 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.258600    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 20:25:15.258761 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.258634    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 20:25:15.258761 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.258673    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zhrfm\""
Sep 08 20:25:15.260215 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.260199    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.262528 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.262510    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.262626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.262589    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 20:25:15.263018 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.262965    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 20:25:15.263018 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.263004    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 20:25:15.263124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.263039    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"
Sep 08 20:25:15.263124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.263070    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t2w4f\""
Sep 08 20:25:15.265006 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.264986    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.265280 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.265261    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7vdsn\""
Sep 08 20:25:15.265280 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.265273    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 20:25:15.267386 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.267367    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 20:25:15.267479 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.267417    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal"]
Sep 08 20:25:15.267548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.267518    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:15.267723 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.267689    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 20:25:15.267723 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.267708    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:25:15.267866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.267847    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hjnw4\""
Sep 08 20:25:15.267983 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.267956    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 20:25:15.270476 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.270460    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dg2vt\""
Sep 08 20:25:15.270583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.270536    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 20:25:15.270842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.270827    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 20:25:15.276684 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.276654    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 20:25:15.276951 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.276937    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal"]
Sep 08 20:25:15.280536 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.280492    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 20:20:14 +0000 UTC" deadline="2028-03-07 14:28:17.689481065 +0000 UTC"
Sep 08 20:25:15.280536 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.280532    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13098h3m2.408951446s"
Sep 08 20:25:15.290553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.290531    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t5x5m"
Sep 08 20:25:15.299212 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.299189    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t5x5m"
Sep 08 20:25:15.332023 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.330388    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:15.352361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.352344    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 20:25:15.357880 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.357850    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.358020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.357894    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8crm8\" (UniqueName: \"kubernetes.io/projected/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-kube-api-access-8crm8\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.358020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.357946    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6a41838-e8f5-4756-be48-ff1b26dcf96a-agent-certs\") pod \"konnectivity-agent-vnw88\" (UID: \"b6a41838-e8f5-4756-be48-ff1b26dcf96a\") " pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:15.358020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.357974    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rrwk\" (UniqueName: \"kubernetes.io/projected/b3d02a24-aa00-4b8a-a6ca-7390b0909505-kube-api-access-8rrwk\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.358020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.357996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da4ddc69-1d5a-40aa-85d6-52a698185abb-host-slash\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.358235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358020    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-kubelet\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.358235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358046    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-log-socket\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.358235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358139    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-socket-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.358235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358179    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-host\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358216    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ebae549-d876-40d4-a52c-025f6b21dde0-serviceca\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358242    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-system-cni-dir\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358271    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92d5f074-5b53-43a6-b31e-5136411eba01-dbus\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358271    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-host\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358297    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-cni-binary-copy\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358333    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-system-cni-dir\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358394    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6a41838-e8f5-4756-be48-ff1b26dcf96a-konnectivity-ca\") pod \"konnectivity-agent-vnw88\" (UID: \"b6a41838-e8f5-4756-be48-ff1b26dcf96a\") " pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358398    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-socket-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.358478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358439    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358489    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-kubernetes\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358521    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-sys\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358547    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-lib-modules\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358572    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-var-lib-kubelet\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358589    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-tuned\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358544    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-kubernetes\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358587    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-sys\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358643    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-var-lib-kubelet\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358671    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-etc-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358701    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovnkube-script-lib\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358724    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-lib-modules\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358734    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92d5f074-5b53-43a6-b31e-5136411eba01-kubelet-config\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358771    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f938a669-effc-4bb0-8894-fdb5be304643-hosts-file\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358801    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f938a669-effc-4bb0-8894-fdb5be304643-tmp-dir\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358828    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-multus-certs\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.358920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358857    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-device-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358874    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ebae549-d876-40d4-a52c-025f6b21dde0-serviceca\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358885    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-modprobe-d\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358907    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-tmp\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358901    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358950    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-slash\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358943    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-device-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.358974    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-var-lib-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359000    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359017    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-netns\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359037    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-modprobe-d\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359064    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysctl-d\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359101    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-cnibin\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359129    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-socket-dir-parent\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359155    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-daemon-config\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359180    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-etc-selinux\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359187    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysctl-d\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.359724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359203    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-cni-bin\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359232    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359256    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-etc-selinux\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359273    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-node-log\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359313    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-run-ovn-kubernetes\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.359332    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359338    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-cni-bin\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359364    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-cni-multus\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.359391    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs podName:f559af35-ddd5-4034-93fb-94b0e072bbd0 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:15.859370067 +0000 UTC m=+2.024540652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs") pod "network-metrics-daemon-qxhpg" (UID: "f559af35-ddd5-4034-93fb-94b0e072bbd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359459    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359486    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-systemd-units\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359506    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-cni-netd\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359526    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-conf-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359546    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-run-netns\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359583    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359590    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-env-overrides\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.360691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-sys-fs\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359669    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cni-binary-copy\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359692    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359716    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-system-cni-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359734    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-sys-fs\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359832    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-k8s-cni-cncf-io\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359864    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g77h\" (UniqueName: \"kubernetes.io/projected/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-kube-api-access-9g77h\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359886    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8l62\" (UniqueName: \"kubernetes.io/projected/f559af35-ddd5-4034-93fb-94b0e072bbd0-kube-api-access-q8l62\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359930    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzq26\" (UniqueName: \"kubernetes.io/projected/da4ddc69-1d5a-40aa-85d6-52a698185abb-kube-api-access-hzq26\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359968    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-systemd\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.359994    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-run\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360011    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ebae549-d876-40d4-a52c-025f6b21dde0-host\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360031    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn6wm\" (UniqueName: \"kubernetes.io/projected/2ebae549-d876-40d4-a52c-025f6b21dde0-kube-api-access-bn6wm\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360056    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gstx\" (UniqueName: \"kubernetes.io/projected/f938a669-effc-4bb0-8894-fdb5be304643-kube-api-access-6gstx\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360082    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-kubelet\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360098    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ebae549-d876-40d4-a52c-025f6b21dde0-host\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360109    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqhtm\" (UniqueName: \"kubernetes.io/projected/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-kube-api-access-pqhtm\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.361297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360153    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cnibin\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360195    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-run\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360255    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cni-binary-copy\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360260    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-systemd\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360328    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-cnibin\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360387    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360433    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-cni-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360472    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysconfig\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360502    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360537    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360565    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h866z\" (UniqueName: \"kubernetes.io/projected/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-kube-api-access-h866z\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360594    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysconfig\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360598    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysctl-conf\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360679    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-systemd\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360704    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-ovn\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360728    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovn-node-metrics-cert\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360752    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-hostroot\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.361832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360729    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-sysctl-conf\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.362530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360778    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-etc-kubernetes\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.362530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360816    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovnkube-config\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.362530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360865    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da4ddc69-1d5a-40aa-85d6-52a698185abb-iptables-alerter-script\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.362530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.360943    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-registration-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.362530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.361061    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-registration-dir\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.362530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.361101    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.362530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.362200    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-etc-tuned\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.362530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.362239    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-tmp\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.373493 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.373471    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:15.373493 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.373492    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:15.373672 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.373502    2561 projected.go:196] Error preparing data for projected volume kube-api-access-wcb2v for pod openshift-network-diagnostics/network-check-target-kslns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:15.373672 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.373557    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v podName:7b512a7f-cf0f-44bc-8689-713e89afb601 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:15.873542595 +0000 UTC m=+2.038713153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wcb2v" (UniqueName: "kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v") pod "network-check-target-kslns" (UID: "7b512a7f-cf0f-44bc-8689-713e89afb601") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:15.375775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.375747    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8crm8\" (UniqueName: \"kubernetes.io/projected/ee07fbc3-149f-43cb-8d58-4cc6aef52bce-kube-api-access-8crm8\") pod \"multus-additional-cni-plugins-4vfsr\" (UID: \"ee07fbc3-149f-43cb-8d58-4cc6aef52bce\") " pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.377235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.377211    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqhtm\" (UniqueName: \"kubernetes.io/projected/5d69e2ba-84fe-4d1f-a10c-7f7a43a89920-kube-api-access-pqhtm\") pod \"aws-ebs-csi-driver-node-vgftq\" (UID: \"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.377953 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.377930    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn6wm\" (UniqueName: \"kubernetes.io/projected/2ebae549-d876-40d4-a52c-025f6b21dde0-kube-api-access-bn6wm\") pod \"node-ca-sktlj\" (UID: \"2ebae549-d876-40d4-a52c-025f6b21dde0\") " pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.378445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.378428    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8l62\" (UniqueName: \"kubernetes.io/projected/f559af35-ddd5-4034-93fb-94b0e072bbd0-kube-api-access-q8l62\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:15.379135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.379117    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g77h\" (UniqueName: \"kubernetes.io/projected/2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9-kube-api-access-9g77h\") pod \"tuned-5b5hz\" (UID: \"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9\") " pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.461392 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-cni-bin\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461404    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-node-log\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-run-ovn-kubernetes\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461449    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-cni-bin\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461464    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-cni-bin\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461469    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-node-log\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461472    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-cni-multus\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461509    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-run-ovn-kubernetes\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461513    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-systemd-units\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461535    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-systemd-units\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461552    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-cni-netd\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461576    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-conf-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461598    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-run-netns\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461615    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-env-overrides\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461621    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-cni-netd\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461642    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-cni-bin\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-system-cni-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461598    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-cni-multus\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.461678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461675    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-k8s-cni-cncf-io\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461675    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-run-netns\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461704    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-conf-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461715    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-k8s-cni-cncf-io\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461734    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-system-cni-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461751    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzq26\" (UniqueName: \"kubernetes.io/projected/da4ddc69-1d5a-40aa-85d6-52a698185abb-kube-api-access-hzq26\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461778    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gstx\" (UniqueName: \"kubernetes.io/projected/f938a669-effc-4bb0-8894-fdb5be304643-kube-api-access-6gstx\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461794    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-kubelet\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-cni-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461860    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461881    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-var-lib-kubelet\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461885    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h866z\" (UniqueName: \"kubernetes.io/projected/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-kube-api-access-h866z\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.461960    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-cni-dir\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462007    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462033    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-systemd\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462051    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-ovn\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.462606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462066    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovn-node-metrics-cert\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462102    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-systemd\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462103    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-hostroot\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462118    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-ovn\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.462119    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462138    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-hostroot\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.462192    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret podName:92d5f074-5b53-43a6-b31e-5136411eba01 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:15.962176094 +0000 UTC m=+2.127346654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret") pod "global-pull-secret-syncer-rqs79" (UID: "92d5f074-5b53-43a6-b31e-5136411eba01") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462192    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-env-overrides\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462222    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-etc-kubernetes\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462254    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovnkube-config\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462282    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da4ddc69-1d5a-40aa-85d6-52a698185abb-iptables-alerter-script\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462284    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-etc-kubernetes\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462311    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6a41838-e8f5-4756-be48-ff1b26dcf96a-agent-certs\") pod \"konnectivity-agent-vnw88\" (UID: \"b6a41838-e8f5-4756-be48-ff1b26dcf96a\") " pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rrwk\" (UniqueName: \"kubernetes.io/projected/b3d02a24-aa00-4b8a-a6ca-7390b0909505-kube-api-access-8rrwk\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462359    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da4ddc69-1d5a-40aa-85d6-52a698185abb-host-slash\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462386    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-kubelet\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-log-socket\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.463554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462439    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92d5f074-5b53-43a6-b31e-5136411eba01-dbus\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462466    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-cni-binary-copy\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462490    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6a41838-e8f5-4756-be48-ff1b26dcf96a-konnectivity-ca\") pod \"konnectivity-agent-vnw88\" (UID: \"b6a41838-e8f5-4756-be48-ff1b26dcf96a\") " pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462531    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462575    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-log-socket\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462591    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-etc-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462627    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da4ddc69-1d5a-40aa-85d6-52a698185abb-host-slash\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462691    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92d5f074-5b53-43a6-b31e-5136411eba01-dbus\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462749    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-run-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462789    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-etc-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462828    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-kubelet\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462865    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovnkube-config\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462618    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovnkube-script-lib\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462893    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da4ddc69-1d5a-40aa-85d6-52a698185abb-iptables-alerter-script\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462923    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92d5f074-5b53-43a6-b31e-5136411eba01-kubelet-config\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.462999    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f938a669-effc-4bb0-8894-fdb5be304643-hosts-file\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463026    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f938a669-effc-4bb0-8894-fdb5be304643-tmp-dir\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463052    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-multus-certs\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463000    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92d5f074-5b53-43a6-b31e-5136411eba01-kubelet-config\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463080    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-slash\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463119    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-host-slash\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463120    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-var-lib-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463162    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3d02a24-aa00-4b8a-a6ca-7390b0909505-var-lib-openvswitch\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463156    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-netns\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463198    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovnkube-script-lib\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-cnibin\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463214    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f938a669-effc-4bb0-8894-fdb5be304643-hosts-file\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463244    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-cnibin\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463250    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-socket-dir-parent\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463315    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-socket-dir-parent\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463352    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-multus-certs\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463360    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6a41838-e8f5-4756-be48-ff1b26dcf96a-konnectivity-ca\") pod \"konnectivity-agent-vnw88\" (UID: \"b6a41838-e8f5-4756-be48-ff1b26dcf96a\") " pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463367    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-daemon-config\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463388    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-host-run-netns\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463488    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-cni-binary-copy\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.463881    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-multus-daemon-config\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.464924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.464262    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f938a669-effc-4bb0-8894-fdb5be304643-tmp-dir\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.465507 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.464647    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3d02a24-aa00-4b8a-a6ca-7390b0909505-ovn-node-metrics-cert\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.465507 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.465195    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6a41838-e8f5-4756-be48-ff1b26dcf96a-agent-certs\") pod \"konnectivity-agent-vnw88\" (UID: \"b6a41838-e8f5-4756-be48-ff1b26dcf96a\") " pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:15.473153 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.473130    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gstx\" (UniqueName: \"kubernetes.io/projected/f938a669-effc-4bb0-8894-fdb5be304643-kube-api-access-6gstx\") pod \"node-resolver-psrxw\" (UID: \"f938a669-effc-4bb0-8894-fdb5be304643\") " pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.473478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.473459    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h866z\" (UniqueName: \"kubernetes.io/projected/97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1-kube-api-access-h866z\") pod \"multus-gtfhv\" (UID: \"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1\") " pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.474106 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.474086    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzq26\" (UniqueName: \"kubernetes.io/projected/da4ddc69-1d5a-40aa-85d6-52a698185abb-kube-api-access-hzq26\") pod \"iptables-alerter-2v5m2\" (UID: \"da4ddc69-1d5a-40aa-85d6-52a698185abb\") " pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.474180 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.474112    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rrwk\" (UniqueName: \"kubernetes.io/projected/b3d02a24-aa00-4b8a-a6ca-7390b0909505-kube-api-access-8rrwk\") pod \"ovnkube-node-5bgdt\" (UID: \"b3d02a24-aa00-4b8a-a6ca-7390b0909505\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.567685 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.567595    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq"
Sep 08 20:25:15.581168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.581135    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5b5hz"
Sep 08 20:25:15.602212 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.602176    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sktlj"
Sep 08 20:25:15.606122 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.606067    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf3209fa1c2452dc88935573fa8e83a6.slice/crio-dfaa34482b6984e4387b2d3b582eb5ac57a11dc70b840ac932d785d86b4eb8f2 WatchSource:0}: Error finding container dfaa34482b6984e4387b2d3b582eb5ac57a11dc70b840ac932d785d86b4eb8f2: Status 404 returned error can't find the container with id dfaa34482b6984e4387b2d3b582eb5ac57a11dc70b840ac932d785d86b4eb8f2
Sep 08 20:25:15.606785 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.606704    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d69e2ba_84fe_4d1f_a10c_7f7a43a89920.slice/crio-d684d608c17d18bcca32511a416030e4ec36123a1b78f97c5d85218ef1ac2af8 WatchSource:0}: Error finding container d684d608c17d18bcca32511a416030e4ec36123a1b78f97c5d85218ef1ac2af8: Status 404 returned error can't find the container with id d684d608c17d18bcca32511a416030e4ec36123a1b78f97c5d85218ef1ac2af8
Sep 08 20:25:15.608999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.608977    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4vfsr"
Sep 08 20:25:15.609602 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.609563    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2bb0b8_2eaa_43ec_bf0f_ebde0320e2b9.slice/crio-b40cce2f56980b38908d855852e472119878b1654242437b814f3543f5d24ba5 WatchSource:0}: Error finding container b40cce2f56980b38908d855852e472119878b1654242437b814f3543f5d24ba5: Status 404 returned error can't find the container with id b40cce2f56980b38908d855852e472119878b1654242437b814f3543f5d24ba5
Sep 08 20:25:15.612986 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.612961    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:25:15.614669 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.614645    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd93909c83a29abfbb61422c635288b0.slice/crio-f7b19acb5ddf4b15cb44ff7a9a3b87b37792c6a1c7f07d8df71df6b8e4538632 WatchSource:0}: Error finding container f7b19acb5ddf4b15cb44ff7a9a3b87b37792c6a1c7f07d8df71df6b8e4538632: Status 404 returned error can't find the container with id f7b19acb5ddf4b15cb44ff7a9a3b87b37792c6a1c7f07d8df71df6b8e4538632
Sep 08 20:25:15.618527 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.618501    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ebae549_d876_40d4_a52c_025f6b21dde0.slice/crio-ed6d6a0e9d1a660c75918c06345979b1865e521a933f390f3a4b6ee11d4a831c WatchSource:0}: Error finding container ed6d6a0e9d1a660c75918c06345979b1865e521a933f390f3a4b6ee11d4a831c: Status 404 returned error can't find the container with id ed6d6a0e9d1a660c75918c06345979b1865e521a933f390f3a4b6ee11d4a831c
Sep 08 20:25:15.621928 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.621899    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee07fbc3_149f_43cb_8d58_4cc6aef52bce.slice/crio-3cf3c3263cc7df1055993a33086f6975bca1b237c95f2b4d1606126adae30853 WatchSource:0}: Error finding container 3cf3c3263cc7df1055993a33086f6975bca1b237c95f2b4d1606126adae30853: Status 404 returned error can't find the container with id 3cf3c3263cc7df1055993a33086f6975bca1b237c95f2b4d1606126adae30853
Sep 08 20:25:15.637211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.637185    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:15.647178 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.647152    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-psrxw"
Sep 08 20:25:15.647479 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.647458    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d02a24_aa00_4b8a_a6ca_7390b0909505.slice/crio-1ac5d98443b81417713ac9e7bf99f1311a087a47dc5b1563311ef8f3fc6b4b6b WatchSource:0}: Error finding container 1ac5d98443b81417713ac9e7bf99f1311a087a47dc5b1563311ef8f3fc6b4b6b: Status 404 returned error can't find the container with id 1ac5d98443b81417713ac9e7bf99f1311a087a47dc5b1563311ef8f3fc6b4b6b
Sep 08 20:25:15.655644 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.655616    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf938a669_effc_4bb0_8894_fdb5be304643.slice/crio-6ce231ca3076edb53cf29b8a45d2c8d74b8f23bdc9c07a45198af091d8d36c81 WatchSource:0}: Error finding container 6ce231ca3076edb53cf29b8a45d2c8d74b8f23bdc9c07a45198af091d8d36c81: Status 404 returned error can't find the container with id 6ce231ca3076edb53cf29b8a45d2c8d74b8f23bdc9c07a45198af091d8d36c81
Sep 08 20:25:15.662949 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.662923    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gtfhv"
Sep 08 20:25:15.671562 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.671535    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97cb2286_ef93_4a2d_bb42_0ec62ab4e2a1.slice/crio-b5808fef9a034780893541cc3a1be97595f46e2b86d1f7cec8ebb6b9e8701ca4 WatchSource:0}: Error finding container b5808fef9a034780893541cc3a1be97595f46e2b86d1f7cec8ebb6b9e8701ca4: Status 404 returned error can't find the container with id b5808fef9a034780893541cc3a1be97595f46e2b86d1f7cec8ebb6b9e8701ca4
Sep 08 20:25:15.672895 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.672815    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2v5m2"
Sep 08 20:25:15.682410 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.682387    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4ddc69_1d5a_40aa_85d6_52a698185abb.slice/crio-a9e4acb86636115847652decebaaaac8d1393eadf93e3d560de89b34aa43eb49 WatchSource:0}: Error finding container a9e4acb86636115847652decebaaaac8d1393eadf93e3d560de89b34aa43eb49: Status 404 returned error can't find the container with id a9e4acb86636115847652decebaaaac8d1393eadf93e3d560de89b34aa43eb49
Sep 08 20:25:15.686713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.686694    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:15.695184 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:15.695165    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a41838_e8f5_4756_be48_ff1b26dcf96a.slice/crio-b240f97fa45f03314a52a75a5e022b942e956f7749e4009104db58b2bbf3d40f WatchSource:0}: Error finding container b240f97fa45f03314a52a75a5e022b942e956f7749e4009104db58b2bbf3d40f: Status 404 returned error can't find the container with id b240f97fa45f03314a52a75a5e022b942e956f7749e4009104db58b2bbf3d40f
Sep 08 20:25:15.696886 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.696866    2561 kuberuntime_manager.go:1459] "Unhandled Error" err="container konnectivity-agent start failed in pod konnectivity-agent-vnw88_kube-system(b6a41838-e8f5-4756-be48-ff1b26dcf96a): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 08 20:25:15.696945 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.696900    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ErrImagePull: \"pull QPS exceeded\"" pod="kube-system/konnectivity-agent-vnw88" podUID="b6a41838-e8f5-4756-be48-ff1b26dcf96a"
Sep 08 20:25:15.866223 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.866130    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:15.866354 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.866317    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:15.866406 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.866397    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs podName:f559af35-ddd5-4034-93fb-94b0e072bbd0 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:16.866379415 +0000 UTC m=+3.031549985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs") pod "network-metrics-daemon-qxhpg" (UID: "f559af35-ddd5-4034-93fb-94b0e072bbd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:15.966891 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.966850    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:15.966891 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:15.966894    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:15.967146 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.967003    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:15.967146 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.967030    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:15.967146 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.967062    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret podName:92d5f074-5b53-43a6-b31e-5136411eba01 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:16.96704724 +0000 UTC m=+3.132217809 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret") pod "global-pull-secret-syncer-rqs79" (UID: "92d5f074-5b53-43a6-b31e-5136411eba01") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:15.967146 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.967053    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:15.967146 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.967098    2561 projected.go:196] Error preparing data for projected volume kube-api-access-wcb2v for pod openshift-network-diagnostics/network-check-target-kslns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:15.967146 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:15.967145    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v podName:7b512a7f-cf0f-44bc-8689-713e89afb601 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:16.96713092 +0000 UTC m=+3.132301498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcb2v" (UniqueName: "kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v") pod "network-check-target-kslns" (UID: "7b512a7f-cf0f-44bc-8689-713e89afb601") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:16.302831 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.302661    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 20:20:15 +0000 UTC" deadline="2028-05-16 21:01:39.689393834 +0000 UTC"
Sep 08 20:25:16.302831 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.302724    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14784h36m23.386675195s"
Sep 08 20:25:16.360563 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.359910    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:16.360563 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.360064    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:16.365875 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.365852    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:16.378191 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.378126    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vnw88" event={"ID":"b6a41838-e8f5-4756-be48-ff1b26dcf96a","Type":"ContainerStarted","Data":"b240f97fa45f03314a52a75a5e022b942e956f7749e4009104db58b2bbf3d40f"}
Sep 08 20:25:16.384561 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.384441    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9018c47455af82aa9aba488b18f1fb9824a505ad46539494602fe936ffc4b601\\\": ErrImagePull: pull QPS exceeded\"" pod="kube-system/konnectivity-agent-vnw88" podUID="b6a41838-e8f5-4756-be48-ff1b26dcf96a"
Sep 08 20:25:16.390543 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.390512    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gtfhv" event={"ID":"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1","Type":"ContainerStarted","Data":"b5808fef9a034780893541cc3a1be97595f46e2b86d1f7cec8ebb6b9e8701ca4"}
Sep 08 20:25:16.406594 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.406552    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" event={"ID":"ee07fbc3-149f-43cb-8d58-4cc6aef52bce","Type":"ContainerStarted","Data":"3cf3c3263cc7df1055993a33086f6975bca1b237c95f2b4d1606126adae30853"}
Sep 08 20:25:16.418067 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.417912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5b5hz" event={"ID":"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9","Type":"ContainerStarted","Data":"b40cce2f56980b38908d855852e472119878b1654242437b814f3543f5d24ba5"}
Sep 08 20:25:16.425751 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.425715    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2v5m2" event={"ID":"da4ddc69-1d5a-40aa-85d6-52a698185abb","Type":"ContainerStarted","Data":"a9e4acb86636115847652decebaaaac8d1393eadf93e3d560de89b34aa43eb49"}
Sep 08 20:25:16.440962 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.440910    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-psrxw" event={"ID":"f938a669-effc-4bb0-8894-fdb5be304643","Type":"ContainerStarted","Data":"6ce231ca3076edb53cf29b8a45d2c8d74b8f23bdc9c07a45198af091d8d36c81"}
Sep 08 20:25:16.448714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.448654    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"1ac5d98443b81417713ac9e7bf99f1311a087a47dc5b1563311ef8f3fc6b4b6b"}
Sep 08 20:25:16.463111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.463072    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sktlj" event={"ID":"2ebae549-d876-40d4-a52c-025f6b21dde0","Type":"ContainerStarted","Data":"ed6d6a0e9d1a660c75918c06345979b1865e521a933f390f3a4b6ee11d4a831c"}
Sep 08 20:25:16.468354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.468310    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal" event={"ID":"dd93909c83a29abfbb61422c635288b0","Type":"ContainerStarted","Data":"f7b19acb5ddf4b15cb44ff7a9a3b87b37792c6a1c7f07d8df71df6b8e4538632"}
Sep 08 20:25:16.477527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.477494    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq" event={"ID":"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920","Type":"ContainerStarted","Data":"d684d608c17d18bcca32511a416030e4ec36123a1b78f97c5d85218ef1ac2af8"}
Sep 08 20:25:16.485492 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.485460    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal" event={"ID":"bf3209fa1c2452dc88935573fa8e83a6","Type":"ContainerStarted","Data":"dfaa34482b6984e4387b2d3b582eb5ac57a11dc70b840ac932d785d86b4eb8f2"}
Sep 08 20:25:16.875168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.875129    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:16.875372 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.875302    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:16.875454 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.875372    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs podName:f559af35-ddd5-4034-93fb-94b0e072bbd0 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:18.875352158 +0000 UTC m=+5.040522731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs") pod "network-metrics-daemon-qxhpg" (UID: "f559af35-ddd5-4034-93fb-94b0e072bbd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:16.976333 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.976288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:16.976552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:16.976354    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:16.976552 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.976508    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:16.976678 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.976582    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret podName:92d5f074-5b53-43a6-b31e-5136411eba01 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:18.976559375 +0000 UTC m=+5.141729936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret") pod "global-pull-secret-syncer-rqs79" (UID: "92d5f074-5b53-43a6-b31e-5136411eba01") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:16.977156 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.977132    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:16.977278 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.977164    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:16.977278 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.977179    2561 projected.go:196] Error preparing data for projected volume kube-api-access-wcb2v for pod openshift-network-diagnostics/network-check-target-kslns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:16.977278 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:16.977226    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v podName:7b512a7f-cf0f-44bc-8689-713e89afb601 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:18.977211066 +0000 UTC m=+5.142381629 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcb2v" (UniqueName: "kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v") pod "network-check-target-kslns" (UID: "7b512a7f-cf0f-44bc-8689-713e89afb601") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:17.304248 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:17.303439    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 20:20:15 +0000 UTC" deadline="2028-03-21 16:25:26.633381112 +0000 UTC"
Sep 08 20:25:17.304248 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:17.303482    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13436h0m9.329903853s"
Sep 08 20:25:17.357184 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:17.357112    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:17.357354 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:17.357274    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:17.357988 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:17.357705    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:17.357988 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:17.357806    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:17.459606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:17.459570    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:18.358081 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:18.358047    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:18.358574 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.358191    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:18.894558 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:18.894517    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:18.894746 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.894684    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:18.894814 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.894756    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs podName:f559af35-ddd5-4034-93fb-94b0e072bbd0 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:22.894735712 +0000 UTC m=+9.059906281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs") pod "network-metrics-daemon-qxhpg" (UID: "f559af35-ddd5-4034-93fb-94b0e072bbd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:18.995675 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:18.995632    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:18.995882 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:18.995702    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:18.995882 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.995864    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:18.996019 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.995947    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret podName:92d5f074-5b53-43a6-b31e-5136411eba01 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:22.995912504 +0000 UTC m=+9.161083083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret") pod "global-pull-secret-syncer-rqs79" (UID: "92d5f074-5b53-43a6-b31e-5136411eba01") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:18.996423 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.996399    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:18.996513 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.996429    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:18.996513 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.996444    2561 projected.go:196] Error preparing data for projected volume kube-api-access-wcb2v for pod openshift-network-diagnostics/network-check-target-kslns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:18.996513 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:18.996494    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v podName:7b512a7f-cf0f-44bc-8689-713e89afb601 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:22.996477179 +0000 UTC m=+9.161647752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcb2v" (UniqueName: "kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v") pod "network-check-target-kslns" (UID: "7b512a7f-cf0f-44bc-8689-713e89afb601") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:19.357918 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:19.357779    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:19.357918 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:19.357812    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:19.358153 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:19.357961    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:19.358153 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:19.358092    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:20.357809 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:20.357325    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:20.357809 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:20.357461    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:21.358244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:21.357800    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:21.358244 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:21.357959    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:21.358244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:21.357993    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:21.358244 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:21.358073    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:22.357877 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:22.357655    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:22.357877 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:22.357800    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:22.930842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:22.930757    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:22.931321 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:22.931002    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:22.931321 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:22.931087    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs podName:f559af35-ddd5-4034-93fb-94b0e072bbd0 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:30.931065576 +0000 UTC m=+17.096236157 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs") pod "network-metrics-daemon-qxhpg" (UID: "f559af35-ddd5-4034-93fb-94b0e072bbd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:23.032935 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:23.032342    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:23.032935 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:23.032402    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:23.032935 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:23.032520    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:23.032935 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:23.032531    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:23.032935 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:23.032557    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:23.032935 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:23.032571    2561 projected.go:196] Error preparing data for projected volume kube-api-access-wcb2v for pod openshift-network-diagnostics/network-check-target-kslns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:23.032935 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:23.032584    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret podName:92d5f074-5b53-43a6-b31e-5136411eba01 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:31.032565314 +0000 UTC m=+17.197735889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret") pod "global-pull-secret-syncer-rqs79" (UID: "92d5f074-5b53-43a6-b31e-5136411eba01") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:23.032935 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:23.032627    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v podName:7b512a7f-cf0f-44bc-8689-713e89afb601 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:31.032610318 +0000 UTC m=+17.197780889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcb2v" (UniqueName: "kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v") pod "network-check-target-kslns" (UID: "7b512a7f-cf0f-44bc-8689-713e89afb601") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:23.357425 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:23.357391    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:23.357425 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:23.357416    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:23.357663 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:23.357551    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:23.357725 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:23.357670    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:24.359544 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:24.359073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:24.359544 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:24.359181    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:25.357706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:25.357672    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:25.357961 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:25.357814    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:25.357961 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:25.357876    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:25.358103 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:25.358020    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:26.357403 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:26.357367    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:26.357890 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:26.357496    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:27.357064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:27.357022    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:27.357261 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:27.357165    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:27.357261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:27.357211    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:27.357383 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:27.357351    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:28.357851 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:28.357813    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:28.358295 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:28.357977    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:29.357123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:29.357061    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:29.357123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:29.357102    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:29.357385 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:29.357199    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:29.357385 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:29.357356    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:30.363591 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:30.363556    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:30.364014 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:30.363669    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:30.993391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:30.993353    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:30.993577 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:30.993498    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:30.993577 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:30.993566    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs podName:f559af35-ddd5-4034-93fb-94b0e072bbd0 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:46.993546395 +0000 UTC m=+33.158716966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs") pod "network-metrics-daemon-qxhpg" (UID: "f559af35-ddd5-4034-93fb-94b0e072bbd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:31.094242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:31.094203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:31.094432 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:31.094263    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:31.094432 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:31.094374    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:31.094557 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:31.094447    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret podName:92d5f074-5b53-43a6-b31e-5136411eba01 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:47.094431392 +0000 UTC m=+33.259601946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret") pod "global-pull-secret-syncer-rqs79" (UID: "92d5f074-5b53-43a6-b31e-5136411eba01") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:31.094557 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:31.094374    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:31.094557 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:31.094490    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:31.094557 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:31.094501    2561 projected.go:196] Error preparing data for projected volume kube-api-access-wcb2v for pod openshift-network-diagnostics/network-check-target-kslns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:31.094761 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:31.094558    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v podName:7b512a7f-cf0f-44bc-8689-713e89afb601 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:47.094532541 +0000 UTC m=+33.259703096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcb2v" (UniqueName: "kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v") pod "network-check-target-kslns" (UID: "7b512a7f-cf0f-44bc-8689-713e89afb601") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:31.357620 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:31.357539    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:31.357799 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:31.357544    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:31.357799 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:31.357657    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:31.357799 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:31.357761    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:32.357673 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:32.357641    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:32.358152 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:32.357757    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:33.357150 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:33.357108    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:33.357358 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:33.357108    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:33.357358 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:33.357267    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:33.357476 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:33.357392    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:34.358152 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:34.357754    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:34.358152 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:34.358091    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:34.538506 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:34.538461    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal" event={"ID":"dd93909c83a29abfbb61422c635288b0","Type":"ContainerStarted","Data":"ec51b7d5b20d79b49c63b0abab53a045ddd33c0e0d11cb4385b310b12cfd42f3"}
Sep 08 20:25:34.540839 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:34.540809    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5b5hz" event={"ID":"2f2bb0b8-2eaa-43ec-bf0f-ebde0320e2b9","Type":"ContainerStarted","Data":"651d82376d3ea4fa8994d9e3f43de616b5f4df7d323b746a07ec660412e85863"}
Sep 08 20:25:34.553740 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:34.553689    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-228.ec2.internal" podStartSLOduration=19.553674504 podStartE2EDuration="19.553674504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:34.55287193 +0000 UTC m=+20.718042506" watchObservedRunningTime="2026-09-08 20:25:34.553674504 +0000 UTC m=+20.718845080"
Sep 08 20:25:34.568381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:34.568331    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5b5hz" podStartSLOduration=1.824076771 podStartE2EDuration="20.568310171s" totalImagesPullingTime="18.7442334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:34.56788396 +0000 UTC m=+20.733054537" watchObservedRunningTime="2026-09-08 20:25:34.568310171 +0000 UTC m=+20.733480749"
Sep 08 20:25:35.357812 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.357657    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:35.357939 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.357733    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:35.357939 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:35.357883    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:35.358029 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:35.358016    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:35.545345 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.545313    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-psrxw" event={"ID":"f938a669-effc-4bb0-8894-fdb5be304643","Type":"ContainerStarted","Data":"2ca7ae3864eb8059ead717445a7aaf4c381ab7ef721ee501a3225bea05e20703"}
Sep 08 20:25:35.548667 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.548628    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"9a17357556e3f07558c3c197820cf9de7af3abab2d948f2e63ac399c1038937a"}
Sep 08 20:25:35.548770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.548729    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"be844e8b1fe89ff976aa97485c87b41d9f9bce66cd52c2312064d94d7cfaffa5"}
Sep 08 20:25:35.548826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.548786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"58e08c528f63e67e2bf1f485cb16d3133325bcb9118283035810d87352b74b9f"}
Sep 08 20:25:35.548873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.548838    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"38aa4e99924b4a16518d20d9d43d09be99db6b5980441aceb0bd333a4d3ce150"}
Sep 08 20:25:35.548873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.548871    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"df1329b931eb4f00786457ec6a35a6a764170e63dfdf5d488206738d85710386"}
Sep 08 20:25:35.550682 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.550346    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sktlj" event={"ID":"2ebae549-d876-40d4-a52c-025f6b21dde0","Type":"ContainerStarted","Data":"d548b87cb62b07957755c9b5f085c2a786f51a12d20d1a28c340faa05c635b62"}
Sep 08 20:25:35.551824 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.551801    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq" event={"ID":"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920","Type":"ContainerStarted","Data":"14bf9f2604a5af78ae127efb72597a7fa1360fb0acb01de676b87e2715aff653"}
Sep 08 20:25:35.553225 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.553203    2561 generic.go:356] "Generic (PLEG): container finished" podID="bf3209fa1c2452dc88935573fa8e83a6" containerID="cb82d6a47ea27eda5c6e4d4b20f81b13f3ac7cfe20f6e44ed54cf99afc2780b4" exitCode=0
Sep 08 20:25:35.553323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.553281    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal" event={"ID":"bf3209fa1c2452dc88935573fa8e83a6","Type":"ContainerDied","Data":"cb82d6a47ea27eda5c6e4d4b20f81b13f3ac7cfe20f6e44ed54cf99afc2780b4"}
Sep 08 20:25:35.556333 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.556311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gtfhv" event={"ID":"97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1","Type":"ContainerStarted","Data":"318716033bc4cddb27e7a907f88d2381794e9d6db291aed0776c1e329f6662b2"}
Sep 08 20:25:35.557826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.557803    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee07fbc3-149f-43cb-8d58-4cc6aef52bce" containerID="f59d4b65be0eb3d705ed1aba91d90aae4bc3f4462a9ac66b09ba9b334e094a14" exitCode=0
Sep 08 20:25:35.557923 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.557844    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" event={"ID":"ee07fbc3-149f-43cb-8d58-4cc6aef52bce","Type":"ContainerDied","Data":"f59d4b65be0eb3d705ed1aba91d90aae4bc3f4462a9ac66b09ba9b334e094a14"}
Sep 08 20:25:35.561439 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.561398    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-psrxw" podStartSLOduration=1.896772398 podStartE2EDuration="20.561385803s" totalImagesPullingTime="18.664613405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:35.560484118 +0000 UTC m=+21.725654695" watchObservedRunningTime="2026-09-08 20:25:35.561385803 +0000 UTC m=+21.726556378"
Sep 08 20:25:35.611567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.611525    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sktlj" podStartSLOduration=2.874756003 podStartE2EDuration="21.61150409s" totalImagesPullingTime="18.736748087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:35.610788448 +0000 UTC m=+21.775959025" watchObservedRunningTime="2026-09-08 20:25:35.61150409 +0000 UTC m=+21.776674666"
Sep 08 20:25:35.630153 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:35.630104    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gtfhv" podStartSLOduration=1.908679405 podStartE2EDuration="20.63008719s" totalImagesPullingTime="18.721407785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:35.629712657 +0000 UTC m=+21.794883234" watchObservedRunningTime="2026-09-08 20:25:35.63008719 +0000 UTC m=+21.795257765"
Sep 08 20:25:36.357602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.357576    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:36.357767 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:36.357682    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:36.384256 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.384233    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 20:25:36.564201 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.564120    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"c040e027e1e6193a8445c7a55bfb31e1eab4f1c509aea818f4335589902e939a"}
Sep 08 20:25:36.565965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.565943    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq" event={"ID":"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920","Type":"ContainerStarted","Data":"b595cba6f3308cb84eda1ec8a6d647f371de05fb6d23811a2b9aa58f1b07559d"}
Sep 08 20:25:36.567779 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.567754    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal" event={"ID":"bf3209fa1c2452dc88935573fa8e83a6","Type":"ContainerStarted","Data":"a5f6dcb6e57fdbb1ce39405c591215ab7f0a62bdc378cc85b488081106c5790a"}
Sep 08 20:25:36.569214 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.569183    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vnw88" event={"ID":"b6a41838-e8f5-4756-be48-ff1b26dcf96a","Type":"ContainerStarted","Data":"25a41130178ecf0274d4e05930c10551e8488d8017098bc1a255136afe1dd8f4"}
Sep 08 20:25:36.570622 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.570599    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2v5m2" event={"ID":"da4ddc69-1d5a-40aa-85d6-52a698185abb","Type":"ContainerStarted","Data":"74cc94dc73fcef3fd2abe77780d9e32b6000c60069bbc8ca6ab671033e011dce"}
Sep 08 20:25:36.583421 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.583387    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-228.ec2.internal" podStartSLOduration=21.583375206 podStartE2EDuration="21.583375206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:36.583216283 +0000 UTC m=+22.748386861" watchObservedRunningTime="2026-09-08 20:25:36.583375206 +0000 UTC m=+22.748545778"
Sep 08 20:25:36.601058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.601002    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2v5m2" podStartSLOduration=2.929627982 podStartE2EDuration="21.600984266s" totalImagesPullingTime="18.671356284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:36.60074051 +0000 UTC m=+22.765911091" watchObservedRunningTime="2026-09-08 20:25:36.600984266 +0000 UTC m=+22.766154845"
Sep 08 20:25:36.614884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.614832    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vnw88" podStartSLOduration=1.863318396 podStartE2EDuration="21.614817335s" totalImagesPullingTime="19.751498939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-08 20:25:36.614650833 +0000 UTC m=+22.779821410" watchObservedRunningTime="2026-09-08 20:25:36.614817335 +0000 UTC m=+22.779987912"
Sep 08 20:25:36.738489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.738459    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:36.739098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:36.739074    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:37.320043 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:37.319929    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T20:25:36.384255071Z","UUID":"cce5180a-f2ae-497a-ba3c-5776810ee584","Handler":null,"Name":"","Endpoint":""}
Sep 08 20:25:37.321906 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:37.321865    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 20:25:37.322038 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:37.321923    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 20:25:37.357277 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:37.357239    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:37.357465 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:37.357387    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:37.357465 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:37.357423    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:37.357583 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:37.357488    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:37.573298 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:37.573201    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:37.573873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:37.573852    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vnw88"
Sep 08 20:25:38.357675 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:38.357639    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:38.357853 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:38.357779    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:38.577632 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:38.577368    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq" event={"ID":"5d69e2ba-84fe-4d1f-a10c-7f7a43a89920","Type":"ContainerStarted","Data":"c70b0fd4a6c0622ac308d7ec6aa8899e444b110e7e679c10645387f946a705a9"}
Sep 08 20:25:38.581196 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:38.581163    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"c02d4d199d4580229f2d16ac4579939fc3ca2a8bf92b8d5c975b970cfaa4b09a"}
Sep 08 20:25:38.594830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:38.594782    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgftq" podStartSLOduration=3.207323344 podStartE2EDuration="24.59476598s" totalImagesPullingTime="21.387442636s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:38.594736947 +0000 UTC m=+24.759907525" watchObservedRunningTime="2026-09-08 20:25:38.59476598 +0000 UTC m=+24.759936549"
Sep 08 20:25:39.357591 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:39.357547    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:39.357755 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:39.357556    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:39.357755 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:39.357692    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:39.357755 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:39.357737    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:40.357085 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:40.356998    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:40.357547 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:40.357113    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:40.590476 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:40.590445    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" event={"ID":"b3d02a24-aa00-4b8a-a6ca-7390b0909505","Type":"ContainerStarted","Data":"dc62ccd7c2294d5be2e16e6d58c9b4a8eaad5f780b20d58fb8ec31460ebb36b0"}
Sep 08 20:25:40.600308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:40.600264    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:40.618539 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:40.618507    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:40.627906 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:40.627798    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt" podStartSLOduration=7.7279065540000005 podStartE2EDuration="26.627781652s" totalImagesPullingTime="18.899875098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:40.627630801 +0000 UTC m=+26.792801379" watchObservedRunningTime="2026-09-08 20:25:40.627781652 +0000 UTC m=+26.792952231"
Sep 08 20:25:41.357446 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:41.357255    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:41.357954 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:41.357258    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:41.357954 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:41.357516    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:41.357954 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:41.357600    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:41.594583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:41.594547    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee07fbc3-149f-43cb-8d58-4cc6aef52bce" containerID="f56cef185b6d012e78dc0dd6d819f01b039947295f951195dece9ce3689f1a6c" exitCode=0
Sep 08 20:25:41.594768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:41.594629    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" event={"ID":"ee07fbc3-149f-43cb-8d58-4cc6aef52bce","Type":"ContainerDied","Data":"f56cef185b6d012e78dc0dd6d819f01b039947295f951195dece9ce3689f1a6c"}
Sep 08 20:25:41.594768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:41.594723    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 20:25:41.595131 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:41.595110    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:41.611467 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:41.611359    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:42.357474 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:42.357439    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:42.358116 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:42.357563    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:42.400694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:42.400658    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qxhpg"]
Sep 08 20:25:42.400865 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:42.400810    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:42.400977 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:42.400945    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:42.404604 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:42.404578    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqs79"]
Sep 08 20:25:42.407322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:42.407066    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kslns"]
Sep 08 20:25:42.407322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:42.407188    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:42.407322 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:42.407299    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:42.596611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:42.596580    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:42.596762 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:42.596647    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 20:25:42.596762 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:42.596706    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:43.601243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:43.601210    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee07fbc3-149f-43cb-8d58-4cc6aef52bce" containerID="2466ca98d6f3bbbe0e08208c6f831d224be094f5c8497ef82a77ad9ec5208ff4" exitCode=0
Sep 08 20:25:43.601691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:43.601260    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" event={"ID":"ee07fbc3-149f-43cb-8d58-4cc6aef52bce","Type":"ContainerDied","Data":"2466ca98d6f3bbbe0e08208c6f831d224be094f5c8497ef82a77ad9ec5208ff4"}
Sep 08 20:25:43.601691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:43.601476    2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 20:25:44.358463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:44.358387    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:44.358619 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:44.358462    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:44.358619 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:44.358465    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:44.358619 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:44.358575    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:44.358619 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:44.358575    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:44.358797 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:44.358669    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:44.642724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:44.642647    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:25:45.608270 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:45.608239    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee07fbc3-149f-43cb-8d58-4cc6aef52bce" containerID="3e934fbc0d4f7a0e9896eae7ea71d401179a8e876fa3f4d5a30574feb04071f4" exitCode=0
Sep 08 20:25:45.608433 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:45.608281    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" event={"ID":"ee07fbc3-149f-43cb-8d58-4cc6aef52bce","Type":"ContainerDied","Data":"3e934fbc0d4f7a0e9896eae7ea71d401179a8e876fa3f4d5a30574feb04071f4"}
Sep 08 20:25:46.357861 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:46.357824    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:46.358330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:46.357825    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:46.358330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:46.357976    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:46.358330 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:46.357992    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:46.358330 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:46.358109    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:46.358330 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:46.358168    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:47.007271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:47.007240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:47.007509 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:47.007379    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:47.007509 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:47.007439    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs podName:f559af35-ddd5-4034-93fb-94b0e072bbd0 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.007423995 +0000 UTC m=+65.172594566 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs") pod "network-metrics-daemon-qxhpg" (UID: "f559af35-ddd5-4034-93fb-94b0e072bbd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:47.107974 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:47.107939    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:47.108161 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:47.107981    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:47.108161 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:47.108083    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:47.108161 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:47.108105    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:47.108161 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:47.108115    2561 projected.go:196] Error preparing data for projected volume kube-api-access-wcb2v for pod openshift-network-diagnostics/network-check-target-kslns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:47.108161 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:47.108128    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:47.108161 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:47.108162    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v podName:7b512a7f-cf0f-44bc-8689-713e89afb601 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.108149677 +0000 UTC m=+65.273320263 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcb2v" (UniqueName: "kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v") pod "network-check-target-kslns" (UID: "7b512a7f-cf0f-44bc-8689-713e89afb601") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:47.108395 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:47.108177    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret podName:92d5f074-5b53-43a6-b31e-5136411eba01 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.108169937 +0000 UTC m=+65.273340491 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret") pod "global-pull-secret-syncer-rqs79" (UID: "92d5f074-5b53-43a6-b31e-5136411eba01") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:48.357872 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.357782    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:48.357872 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.357808    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:48.358328 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.357788    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:48.358328 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:48.357927    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kslns" podUID="7b512a7f-cf0f-44bc-8689-713e89afb601"
Sep 08 20:25:48.358328 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:48.358034    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qxhpg" podUID="f559af35-ddd5-4034-93fb-94b0e072bbd0"
Sep 08 20:25:48.358328 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:48.358132    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqs79" podUID="92d5f074-5b53-43a6-b31e-5136411eba01"
Sep 08 20:25:48.628247 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.628177    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-228.ec2.internal" event="NodeReady"
Sep 08 20:25:48.628398 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.628341    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 20:25:48.674582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.674546    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c6674f4b8-n8ntr"]
Sep 08 20:25:48.678218 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.678190    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.682830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.682794    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nwnm8\""
Sep 08 20:25:48.682830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.682824    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 08 20:25:48.683088 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.682948    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 08 20:25:48.684179 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.684158    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 08 20:25:48.690828 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.690800    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 08 20:25:48.691163 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.691141    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xmz24"]
Sep 08 20:25:48.695131 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.695112    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.697475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.697453    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vbwpr\""
Sep 08 20:25:48.697584 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.697493    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 20:25:48.697941 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.697923    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 20:25:48.702880 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.702861    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j69nw"]
Sep 08 20:25:48.706037 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.705985    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j69nw"
Sep 08 20:25:48.706602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.706580    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c6674f4b8-n8ntr"]
Sep 08 20:25:48.707326 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.707305    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xmz24"]
Sep 08 20:25:48.709457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.709428    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 20:25:48.709551 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.709434    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t62jz\""
Sep 08 20:25:48.709551 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.709462    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 20:25:48.709551 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.709466    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 20:25:48.720034 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.720012    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j69nw"]
Sep 08 20:25:48.819978 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.819937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74e19476-d6be-47fc-a532-4129d15638af-cert\") pod \"ingress-canary-j69nw\" (UID: \"74e19476-d6be-47fc-a532-4129d15638af\") " pod="openshift-ingress-canary/ingress-canary-j69nw"
Sep 08 20:25:48.819978 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.819983    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxq6k\" (UniqueName: \"kubernetes.io/projected/74e19476-d6be-47fc-a532-4129d15638af-kube-api-access-zxq6k\") pod \"ingress-canary-j69nw\" (UID: \"74e19476-d6be-47fc-a532-4129d15638af\") " pod="openshift-ingress-canary/ingress-canary-j69nw"
Sep 08 20:25:48.820226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820018    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-trusted-ca\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.820226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820044    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-bound-sa-token\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.820226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820084    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-installation-pull-secrets\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.820226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820108    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bwsk\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-kube-api-access-7bwsk\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.820226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820136    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.820226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820170    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-image-registry-private-configuration\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.820226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820188    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb4283e3-b5ae-4809-8bee-8a0320e57de4-ca-trust-extracted\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.820226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820224    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-certificates\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.820619 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820239    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-config-volume\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.820619 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820275    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.820619 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820352    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-tmp-dir\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.820619 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.820392    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np9f2\" (UniqueName: \"kubernetes.io/projected/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-kube-api-access-np9f2\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.920739 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.920698    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.920764    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-tmp-dir\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.920789    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np9f2\" (UniqueName: \"kubernetes.io/projected/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-kube-api-access-np9f2\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.920819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74e19476-d6be-47fc-a532-4129d15638af-cert\") pod \"ingress-canary-j69nw\" (UID: \"74e19476-d6be-47fc-a532-4129d15638af\") " pod="openshift-ingress-canary/ingress-canary-j69nw"
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.920845    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxq6k\" (UniqueName: \"kubernetes.io/projected/74e19476-d6be-47fc-a532-4129d15638af-kube-api-access-zxq6k\") pod \"ingress-canary-j69nw\" (UID: \"74e19476-d6be-47fc-a532-4129d15638af\") " pod="openshift-ingress-canary/ingress-canary-j69nw"
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:48.920869    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.920882    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-trusted-ca\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:48.920908    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6674f4b8-n8ntr: secret "image-registry-tls" not found
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.920923    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-bound-sa-token\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.920958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.920951    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-installation-pull-secrets\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:48.920974    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls podName:cb4283e3-b5ae-4809-8bee-8a0320e57de4 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:49.420953155 +0000 UTC m=+35.586123733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls") pod "image-registry-7c6674f4b8-n8ntr" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4") : secret "image-registry-tls" not found
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921011    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bwsk\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-kube-api-access-7bwsk\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921046    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921090    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-image-registry-private-configuration\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921115    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb4283e3-b5ae-4809-8bee-8a0320e57de4-ca-trust-extracted\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921140    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-certificates\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921160    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-config-volume\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921171    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-tmp-dir\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:48.921194    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:25:48.921430 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:48.921269    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls podName:68bd9156-10b5-407c-8ef9-3ce4d0e0204b nodeName:}" failed. No retries permitted until 2026-09-08 20:25:49.421248551 +0000 UTC m=+35.586419118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls") pod "dns-default-xmz24" (UID: "68bd9156-10b5-407c-8ef9-3ce4d0e0204b") : secret "dns-default-metrics-tls" not found
Sep 08 20:25:48.921970 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921742    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-config-volume\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.921970 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.921888    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-certificates\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.922151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.922127    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb4283e3-b5ae-4809-8bee-8a0320e57de4-ca-trust-extracted\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.922589 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.922566    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-trusted-ca\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.925510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.925483    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-installation-pull-secrets\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.925607 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.925491    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-image-registry-private-configuration\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.925607 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.925497    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74e19476-d6be-47fc-a532-4129d15638af-cert\") pod \"ingress-canary-j69nw\" (UID: \"74e19476-d6be-47fc-a532-4129d15638af\") " pod="openshift-ingress-canary/ingress-canary-j69nw"
Sep 08 20:25:48.933194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.933170    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np9f2\" (UniqueName: \"kubernetes.io/projected/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-kube-api-access-np9f2\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:48.934111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.934090    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bwsk\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-kube-api-access-7bwsk\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.934658 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.934635    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-bound-sa-token\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:48.935354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.935326    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxq6k\" (UniqueName: \"kubernetes.io/projected/74e19476-d6be-47fc-a532-4129d15638af-kube-api-access-zxq6k\") pod \"ingress-canary-j69nw\" (UID: \"74e19476-d6be-47fc-a532-4129d15638af\") " pod="openshift-ingress-canary/ingress-canary-j69nw"
Sep 08 20:25:48.973131 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:48.973090    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnw88_b6a41838-e8f5-4756-be48-ff1b26dcf96a/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:49.017191 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:49.017151    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j69nw"
Sep 08 20:25:49.424453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:49.424415    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:49.424953 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:49.424473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:49.424953 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:49.424579    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:25:49.424953 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:49.424659    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls podName:68bd9156-10b5-407c-8ef9-3ce4d0e0204b nodeName:}" failed. No retries permitted until 2026-09-08 20:25:50.424634493 +0000 UTC m=+36.589805055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls") pod "dns-default-xmz24" (UID: "68bd9156-10b5-407c-8ef9-3ce4d0e0204b") : secret "dns-default-metrics-tls" not found
Sep 08 20:25:49.424953 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:49.424580    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:25:49.424953 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:49.424686    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6674f4b8-n8ntr: secret "image-registry-tls" not found
Sep 08 20:25:49.424953 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:49.424738    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls podName:cb4283e3-b5ae-4809-8bee-8a0320e57de4 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:50.424723707 +0000 UTC m=+36.589894282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls") pod "image-registry-7c6674f4b8-n8ntr" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4") : secret "image-registry-tls" not found
Sep 08 20:25:50.314123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.313819    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j69nw"]
Sep 08 20:25:50.329956 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:25:50.329919    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e19476_d6be_47fc_a532_4129d15638af.slice/crio-7a16e8233dbedc24720c68a3ff48621f2f94f8852053a6e16aa7b296b570d80f WatchSource:0}: Error finding container 7a16e8233dbedc24720c68a3ff48621f2f94f8852053a6e16aa7b296b570d80f: Status 404 returned error can't find the container with id 7a16e8233dbedc24720c68a3ff48621f2f94f8852053a6e16aa7b296b570d80f
Sep 08 20:25:50.357694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.357613    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:25:50.357694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.357632    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:25:50.357928 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.357798    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:25:50.360745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.360725    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 20:25:50.360862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.360727    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 20:25:50.361503 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.361477    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kbhtv\""
Sep 08 20:25:50.361602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.361483    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 20:25:50.361646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.361625    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 20:25:50.361740 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.361711    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2nzg7\""
Sep 08 20:25:50.433198 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.433157    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:50.433599 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.433229    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:50.433599 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:50.433317    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:25:50.433599 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:50.433369    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:25:50.433599 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:50.433386    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls podName:68bd9156-10b5-407c-8ef9-3ce4d0e0204b nodeName:}" failed. No retries permitted until 2026-09-08 20:25:52.433371892 +0000 UTC m=+38.598542479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls") pod "dns-default-xmz24" (UID: "68bd9156-10b5-407c-8ef9-3ce4d0e0204b") : secret "dns-default-metrics-tls" not found
Sep 08 20:25:50.433599 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:50.433387    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6674f4b8-n8ntr: secret "image-registry-tls" not found
Sep 08 20:25:50.433599 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:50.433417    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls podName:cb4283e3-b5ae-4809-8bee-8a0320e57de4 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:52.433410057 +0000 UTC m=+38.598580642 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls") pod "image-registry-7c6674f4b8-n8ntr" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4") : secret "image-registry-tls" not found
Sep 08 20:25:50.622716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.622633    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee07fbc3-149f-43cb-8d58-4cc6aef52bce" containerID="7b66addc1b2c3fdf4cbcbd1abbe96dd5da2db5896b34db768b8b2dabd42aadf6" exitCode=0
Sep 08 20:25:50.622857 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.622710    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" event={"ID":"ee07fbc3-149f-43cb-8d58-4cc6aef52bce","Type":"ContainerDied","Data":"7b66addc1b2c3fdf4cbcbd1abbe96dd5da2db5896b34db768b8b2dabd42aadf6"}
Sep 08 20:25:50.623908 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:50.623858    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j69nw" event={"ID":"74e19476-d6be-47fc-a532-4129d15638af","Type":"ContainerStarted","Data":"7a16e8233dbedc24720c68a3ff48621f2f94f8852053a6e16aa7b296b570d80f"}
Sep 08 20:25:51.630560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:51.630527    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee07fbc3-149f-43cb-8d58-4cc6aef52bce" containerID="40a7758ec4b0efc6cf4429203fbd32111602eab4dc73dc7482e0e703ba80aa6f" exitCode=0
Sep 08 20:25:51.630990 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:51.630575    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" event={"ID":"ee07fbc3-149f-43cb-8d58-4cc6aef52bce","Type":"ContainerDied","Data":"40a7758ec4b0efc6cf4429203fbd32111602eab4dc73dc7482e0e703ba80aa6f"}
Sep 08 20:25:52.449862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:52.449818    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:52.450065 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:52.449936    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:52.450065 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:52.450015    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:25:52.450168 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:52.450092    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:25:52.450168 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:52.450115    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6674f4b8-n8ntr: secret "image-registry-tls" not found
Sep 08 20:25:52.450168 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:52.450096    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls podName:68bd9156-10b5-407c-8ef9-3ce4d0e0204b nodeName:}" failed. No retries permitted until 2026-09-08 20:25:56.450074709 +0000 UTC m=+42.615245281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls") pod "dns-default-xmz24" (UID: "68bd9156-10b5-407c-8ef9-3ce4d0e0204b") : secret "dns-default-metrics-tls" not found
Sep 08 20:25:52.450336 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:52.450173    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls podName:cb4283e3-b5ae-4809-8bee-8a0320e57de4 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:56.450154024 +0000 UTC m=+42.615324585 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls") pod "image-registry-7c6674f4b8-n8ntr" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4") : secret "image-registry-tls" not found
Sep 08 20:25:52.636858 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:52.636821    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" event={"ID":"ee07fbc3-149f-43cb-8d58-4cc6aef52bce","Type":"ContainerStarted","Data":"75e742102a125a465b1a3c641fe8b9c813456eb087173554dc2d645ae2f00589"}
Sep 08 20:25:52.661062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:52.661013    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4vfsr" podStartSLOduration=8.094285294 podStartE2EDuration="38.660998301s" totalImagesPullingTime="30.566713007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:52.66057844 +0000 UTC m=+38.825749019" watchObservedRunningTime="2026-09-08 20:25:52.660998301 +0000 UTC m=+38.826168877"
Sep 08 20:25:53.640535 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:53.640493    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j69nw" event={"ID":"74e19476-d6be-47fc-a532-4129d15638af","Type":"ContainerStarted","Data":"ba4a78177ff79c2448134b573baea92be9c873f620532546e5602c25942f616d"}
Sep 08 20:25:53.656268 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:53.656211    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j69nw" podStartSLOduration=2.984266654 podStartE2EDuration="5.656190453s" totalImagesPullingTime="2.671923799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:53.656102013 +0000 UTC m=+39.821272595" watchObservedRunningTime="2026-09-08 20:25:53.656190453 +0000 UTC m=+39.821361030"
Sep 08 20:25:56.479397 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:56.479349    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:25:56.479772 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:25:56.479426    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:25:56.479772 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:56.479517    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:25:56.479772 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:56.479579    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls podName:68bd9156-10b5-407c-8ef9-3ce4d0e0204b nodeName:}" failed. No retries permitted until 2026-09-08 20:26:04.47956467 +0000 UTC m=+50.644735224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls") pod "dns-default-xmz24" (UID: "68bd9156-10b5-407c-8ef9-3ce4d0e0204b") : secret "dns-default-metrics-tls" not found
Sep 08 20:25:56.479772 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:56.479590    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:25:56.479772 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:56.479611    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6674f4b8-n8ntr: secret "image-registry-tls" not found
Sep 08 20:25:56.479772 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:25:56.479678    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls podName:cb4283e3-b5ae-4809-8bee-8a0320e57de4 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:04.479663817 +0000 UTC m=+50.644834391 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls") pod "image-registry-7c6674f4b8-n8ntr" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4") : secret "image-registry-tls" not found
Sep 08 20:26:04.541593 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:04.541551    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:26:04.542046 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:04.541617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:26:04.542046 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:04.541711    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:04.542046 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:04.541784    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls podName:68bd9156-10b5-407c-8ef9-3ce4d0e0204b nodeName:}" failed. No retries permitted until 2026-09-08 20:26:20.541768303 +0000 UTC m=+66.706938862 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls") pod "dns-default-xmz24" (UID: "68bd9156-10b5-407c-8ef9-3ce4d0e0204b") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:04.542046 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:04.541714    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:26:04.542046 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:04.541812    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6674f4b8-n8ntr: secret "image-registry-tls" not found
Sep 08 20:26:04.542046 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:04.541851    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls podName:cb4283e3-b5ae-4809-8bee-8a0320e57de4 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:20.541839882 +0000 UTC m=+66.707010461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls") pod "image-registry-7c6674f4b8-n8ntr" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4") : secret "image-registry-tls" not found
Sep 08 20:26:14.653121 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:14.653091    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bgdt"
Sep 08 20:26:19.055674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.055630    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:26:19.058448 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.058424    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 20:26:19.066655 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.066624    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 20:26:19.066800 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.066715    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs podName:f559af35-ddd5-4034-93fb-94b0e072bbd0 nodeName:}" failed. No retries permitted until 2026-09-08 20:27:23.066692218 +0000 UTC m=+129.231862786 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs") pod "network-metrics-daemon-qxhpg" (UID: "f559af35-ddd5-4034-93fb-94b0e072bbd0") : secret "metrics-daemon-secret" not found
Sep 08 20:26:19.148491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.148455    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5c98988766-lv6fl"]
Sep 08 20:26:19.153868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.153845    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.156348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.156329    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-stats-auth\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.156459 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.156363    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.156459 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.156381    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdl5c\" (UniqueName: \"kubernetes.io/projected/350750d6-ab60-46cc-9421-cdb689c07574-kube-api-access-wdl5c\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.156459 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.156415    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:26:19.156595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.156483    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:26:19.156595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.156539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-default-certificate\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.156667 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.156614    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.157419 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.157400    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 08 20:26:19.157617 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.157595    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.157873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.157851    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gzm5w\""
Sep 08 20:26:19.157994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.157969    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 08 20:26:19.157994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.157987    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 08 20:26:19.158114 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.158020    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.158114 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.158024    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 08 20:26:19.158597 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.158584    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.158920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.158889    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 20:26:19.169171 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.169154    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.170015 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.169994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92d5f074-5b53-43a6-b31e-5136411eba01-original-pull-secret\") pod \"global-pull-secret-syncer-rqs79\" (UID: \"92d5f074-5b53-43a6-b31e-5136411eba01\") " pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:26:19.170105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.170091    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqs79"
Sep 08 20:26:19.180114 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.179948    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcb2v\" (UniqueName: \"kubernetes.io/projected/7b512a7f-cf0f-44bc-8689-713e89afb601-kube-api-access-wcb2v\") pod \"network-check-target-kslns\" (UID: \"7b512a7f-cf0f-44bc-8689-713e89afb601\") " pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:26:19.182045 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.181985    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c98988766-lv6fl"]
Sep 08 20:26:19.244458 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.244422    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-tnls4"]
Sep 08 20:26:19.248110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.248088    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-h4c88"]
Sep 08 20:26:19.248242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.248203    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:19.250985 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.250838    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-r6jxm\""
Sep 08 20:26:19.250985 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.250881    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 08 20:26:19.250985 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.250915    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 08 20:26:19.252117 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.252103    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.256947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.256925    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-stats-auth\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.257058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.256956    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be92c86c-f31f-43e5-94a5-66d0d8dc4415-config\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.257058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.256973    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be92c86c-f31f-43e5-94a5-66d0d8dc4415-serving-cert\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.257058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257001    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.257215 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.257124    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.757104802 +0000 UTC m=+65.922275381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : configmap references non-existent config key: service-ca.crt
Sep 08 20:26:19.257215 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257133    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 08 20:26:19.257215 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257169    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdl5c\" (UniqueName: \"kubernetes.io/projected/350750d6-ab60-46cc-9421-cdb689c07574-kube-api-access-wdl5c\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.257388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257233    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:19.257388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257181    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pj74h\""
Sep 08 20:26:19.257388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257264    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be92c86c-f31f-43e5-94a5-66d0d8dc4415-trusted-ca\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.257388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257323    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2wsr\" (UniqueName: \"kubernetes.io/projected/be92c86c-f31f-43e5-94a5-66d0d8dc4415-kube-api-access-k2wsr\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.257550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257406    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-default-certificate\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.257550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257444    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-nginx-conf\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:19.257550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.257660 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.257566    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 20:26:19.257660 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.257627    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.757609699 +0000 UTC m=+65.922780267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : secret "router-metrics-certs-default" not found
Sep 08 20:26:19.257918 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.257884    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 08 20:26:19.258090 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.258073    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.258232 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.258215    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.259581 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.259564    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-stats-auth\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.259822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.259791    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-default-certificate\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.260824 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.260801    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-tnls4"]
Sep 08 20:26:19.266577 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.266540    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 08 20:26:19.266994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.266977    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-h4c88"]
Sep 08 20:26:19.276947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.276927    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdl5c\" (UniqueName: \"kubernetes.io/projected/350750d6-ab60-46cc-9421-cdb689c07574-kube-api-access-wdl5c\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.332136 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.332067    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"]
Sep 08 20:26:19.336005 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.335982    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.338846 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.338828    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 08 20:26:19.338996 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.338828    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 08 20:26:19.338996 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.338959    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.339098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.339084    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-n9d5s\""
Sep 08 20:26:19.339142 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.339111    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.351743 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.351721    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"]
Sep 08 20:26:19.358103 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358082    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-nginx-conf\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:19.358242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358132    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be92c86c-f31f-43e5-94a5-66d0d8dc4415-config\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.358242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358150    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be92c86c-f31f-43e5-94a5-66d0d8dc4415-serving-cert\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.358242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358169    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9733443-6da4-4bd0-99c1-0115e2aa4623-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.358242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358199    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:19.358242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be92c86c-f31f-43e5-94a5-66d0d8dc4415-trusted-ca\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.358242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358239    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2wsr\" (UniqueName: \"kubernetes.io/projected/be92c86c-f31f-43e5-94a5-66d0d8dc4415-kube-api-access-k2wsr\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.358550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358268    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9733443-6da4-4bd0-99c1-0115e2aa4623-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.358550 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.358312    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 20:26:19.358550 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.358378    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert podName:7d8ba035-e6a4-4e15-9a1e-79b7e6742b41 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.858358446 +0000 UTC m=+66.023529014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tnls4" (UID: "7d8ba035-e6a4-4e15-9a1e-79b7e6742b41") : secret "networking-console-plugin-cert" not found
Sep 08 20:26:19.358550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358414    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5jjs\" (UniqueName: \"kubernetes.io/projected/a9733443-6da4-4bd0-99c1-0115e2aa4623-kube-api-access-v5jjs\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.358868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358847    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be92c86c-f31f-43e5-94a5-66d0d8dc4415-config\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.358952 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.358868    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-nginx-conf\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:19.359129 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.359111    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be92c86c-f31f-43e5-94a5-66d0d8dc4415-trusted-ca\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.360538 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.360523    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be92c86c-f31f-43e5-94a5-66d0d8dc4415-serving-cert\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.370119 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.370092    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2wsr\" (UniqueName: \"kubernetes.io/projected/be92c86c-f31f-43e5-94a5-66d0d8dc4415-kube-api-access-k2wsr\") pod \"console-operator-85b454874b-h4c88\" (UID: \"be92c86c-f31f-43e5-94a5-66d0d8dc4415\") " pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.440295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.440264    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqs79"]
Sep 08 20:26:19.443718 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:19.443692    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d5f074_5b53_43a6_b31e_5136411eba01.slice/crio-2f3e8d2da62732b2254d9fc22ad2a596f38975b9b36b7ba330d65264e4206cfb WatchSource:0}: Error finding container 2f3e8d2da62732b2254d9fc22ad2a596f38975b9b36b7ba330d65264e4206cfb: Status 404 returned error can't find the container with id 2f3e8d2da62732b2254d9fc22ad2a596f38975b9b36b7ba330d65264e4206cfb
Sep 08 20:26:19.458757 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.458737    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9733443-6da4-4bd0-99c1-0115e2aa4623-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.458844 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.458794    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9733443-6da4-4bd0-99c1-0115e2aa4623-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.458844 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.458830    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5jjs\" (UniqueName: \"kubernetes.io/projected/a9733443-6da4-4bd0-99c1-0115e2aa4623-kube-api-access-v5jjs\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.459475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.459454    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9733443-6da4-4bd0-99c1-0115e2aa4623-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.460862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.460842    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9733443-6da4-4bd0-99c1-0115e2aa4623-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.466962 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.466943    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5jjs\" (UniqueName: \"kubernetes.io/projected/a9733443-6da4-4bd0-99c1-0115e2aa4623-kube-api-access-v5jjs\") pod \"kube-storage-version-migrator-operator-8d7dd4675-fl5q8\" (UID: \"a9733443-6da4-4bd0-99c1-0115e2aa4623\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.479356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.479332    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2nzg7\""
Sep 08 20:26:19.487597 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.487575    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:26:19.572491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.572457    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:19.645200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.645170    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"
Sep 08 20:26:19.702805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.702770    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqs79" event={"ID":"92d5f074-5b53-43a6-b31e-5136411eba01","Type":"ContainerStarted","Data":"2f3e8d2da62732b2254d9fc22ad2a596f38975b9b36b7ba330d65264e4206cfb"}
Sep 08 20:26:19.756102 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.756070    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kslns"]
Sep 08 20:26:19.758850 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:19.758821    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b512a7f_cf0f_44bc_8689_713e89afb601.slice/crio-15b975eeaadd91132ab937d6ed68bf523739c5d3e9b1ea34a80a9419ce25aaf9 WatchSource:0}: Error finding container 15b975eeaadd91132ab937d6ed68bf523739c5d3e9b1ea34a80a9419ce25aaf9: Status 404 returned error can't find the container with id 15b975eeaadd91132ab937d6ed68bf523739c5d3e9b1ea34a80a9419ce25aaf9
Sep 08 20:26:19.760581 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.760560    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.760700 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.760674    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:19.760745 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.760699    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:20.760683416 +0000 UTC m=+66.925853970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : configmap references non-existent config key: service-ca.crt
Sep 08 20:26:19.760792 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.760772    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 20:26:19.760916 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.760828    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:20.76081262 +0000 UTC m=+66.925983191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : secret "router-metrics-certs-default" not found
Sep 08 20:26:19.842059 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.841984    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-h4c88"]
Sep 08 20:26:19.844536 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:19.844511    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe92c86c_f31f_43e5_94a5_66d0d8dc4415.slice/crio-5f67afadc60f167728e47c988ad55646ca56696b26232d5dc1010ee3cb522d9a WatchSource:0}: Error finding container 5f67afadc60f167728e47c988ad55646ca56696b26232d5dc1010ee3cb522d9a: Status 404 returned error can't find the container with id 5f67afadc60f167728e47c988ad55646ca56696b26232d5dc1010ee3cb522d9a
Sep 08 20:26:19.861921 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.861884    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:19.862047 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.862032    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 20:26:19.862099 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:19.862090    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert podName:7d8ba035-e6a4-4e15-9a1e-79b7e6742b41 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:20.862074333 +0000 UTC m=+67.027244895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tnls4" (UID: "7d8ba035-e6a4-4e15-9a1e-79b7e6742b41") : secret "networking-console-plugin-cert" not found
Sep 08 20:26:19.913217 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:19.913191    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8"]
Sep 08 20:26:19.915593 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:19.915568    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9733443_6da4_4bd0_99c1_0115e2aa4623.slice/crio-8eb4c97ab9e235358527b1ec45a1bfe3e8121a9adc289e97d3fa6b448916486d WatchSource:0}: Error finding container 8eb4c97ab9e235358527b1ec45a1bfe3e8121a9adc289e97d3fa6b448916486d: Status 404 returned error can't find the container with id 8eb4c97ab9e235358527b1ec45a1bfe3e8121a9adc289e97d3fa6b448916486d
Sep 08 20:26:20.569612 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:20.569568    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:26:20.570138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:20.569675    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:26:20.570138 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.569812    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:26:20.570138 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.569827    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6674f4b8-n8ntr: secret "image-registry-tls" not found
Sep 08 20:26:20.570138 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.569887    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls podName:cb4283e3-b5ae-4809-8bee-8a0320e57de4 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:52.569867267 +0000 UTC m=+98.735037828 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls") pod "image-registry-7c6674f4b8-n8ntr" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4") : secret "image-registry-tls" not found
Sep 08 20:26:20.570426 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.570357    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:20.570426 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.570412    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls podName:68bd9156-10b5-407c-8ef9-3ce4d0e0204b nodeName:}" failed. No retries permitted until 2026-09-08 20:26:52.570396147 +0000 UTC m=+98.735566702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls") pod "dns-default-xmz24" (UID: "68bd9156-10b5-407c-8ef9-3ce4d0e0204b") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:20.708145 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:20.708103    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8" event={"ID":"a9733443-6da4-4bd0-99c1-0115e2aa4623","Type":"ContainerStarted","Data":"8eb4c97ab9e235358527b1ec45a1bfe3e8121a9adc289e97d3fa6b448916486d"}
Sep 08 20:26:20.710358 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:20.710328    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-h4c88" event={"ID":"be92c86c-f31f-43e5-94a5-66d0d8dc4415","Type":"ContainerStarted","Data":"5f67afadc60f167728e47c988ad55646ca56696b26232d5dc1010ee3cb522d9a"}
Sep 08 20:26:20.712457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:20.712429    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kslns" event={"ID":"7b512a7f-cf0f-44bc-8689-713e89afb601","Type":"ContainerStarted","Data":"15b975eeaadd91132ab937d6ed68bf523739c5d3e9b1ea34a80a9419ce25aaf9"}
Sep 08 20:26:20.771686 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:20.771637    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:20.771855 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:20.771743    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:20.771855 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.771794    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 20:26:20.772006 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.771870    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:22.771848243 +0000 UTC m=+68.937018799 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : secret "router-metrics-certs-default" not found
Sep 08 20:26:20.772006 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.771977    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:22.771960592 +0000 UTC m=+68.937131166 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : configmap references non-existent config key: service-ca.crt
Sep 08 20:26:20.873909 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:20.873212    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:20.873909 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.873414    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 20:26:20.873909 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:20.873482    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert podName:7d8ba035-e6a4-4e15-9a1e-79b7e6742b41 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:22.87346272 +0000 UTC m=+69.038633277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tnls4" (UID: "7d8ba035-e6a4-4e15-9a1e-79b7e6742b41") : secret "networking-console-plugin-cert" not found
Sep 08 20:26:22.789448 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:22.789417    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:22.789940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:22.789486    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:22.789940 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:22.789593    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 20:26:22.789940 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:22.789668    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.789650694 +0000 UTC m=+72.954821272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : configmap references non-existent config key: service-ca.crt
Sep 08 20:26:22.789940 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:22.789688    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.789681588 +0000 UTC m=+72.954852178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : secret "router-metrics-certs-default" not found
Sep 08 20:26:22.890444 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:22.890394    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:22.890635 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:22.890547    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 20:26:22.890635 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:22.890612    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert podName:7d8ba035-e6a4-4e15-9a1e-79b7e6742b41 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.890594723 +0000 UTC m=+73.055765292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tnls4" (UID: "7d8ba035-e6a4-4e15-9a1e-79b7e6742b41") : secret "networking-console-plugin-cert" not found
Sep 08 20:26:25.475126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.475090    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kjrd7"]
Sep 08 20:26:25.490468 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.490445    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.492365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.492340    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kjrd7"]
Sep 08 20:26:25.492811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.492786    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 20:26:25.492946 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.492821    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 20:26:25.493515 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.493499    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 20:26:25.493593 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.493535    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4gd8m\""
Sep 08 20:26:25.494046 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.494029    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 20:26:25.510813 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.510782    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbcd2699-0b1b-4611-96dc-18129e3364f8-crio-socket\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.511306 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.510839    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbcd2699-0b1b-4611-96dc-18129e3364f8-data-volume\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.511399 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.511355    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqdsv\" (UniqueName: \"kubernetes.io/projected/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-api-access-nqdsv\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.511473 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.511402    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.511549 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.511528    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.612118 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.612069    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.612184 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.612124    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbcd2699-0b1b-4611-96dc-18129e3364f8-crio-socket\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.612184 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.612166    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbcd2699-0b1b-4611-96dc-18129e3364f8-data-volume\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.612284 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.612193    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqdsv\" (UniqueName: \"kubernetes.io/projected/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-api-access-nqdsv\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.612284 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:25.612196    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:25.612284 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.612217    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.612284 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:25.612257    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.112235366 +0000 UTC m=+72.277405923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:25.612284 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:25.612281    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:25.612457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.612271    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbcd2699-0b1b-4611-96dc-18129e3364f8-crio-socket\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.612457 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:25.612321    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.112306302 +0000 UTC m=+72.277476873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:25.612457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.612447    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbcd2699-0b1b-4611-96dc-18129e3364f8-data-volume\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:25.629635 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:25.629600    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqdsv\" (UniqueName: \"kubernetes.io/projected/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-api-access-nqdsv\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:26.116563 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.116528    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:26.116782 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.116592    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:26.116782 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.116672    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:26.116782 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.116748    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:26.116930 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.116753    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:27.11673308 +0000 UTC m=+73.281903657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:26.116930 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.116812    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:27.11679577 +0000 UTC m=+73.281966327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:26.732094 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.732055    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8" event={"ID":"a9733443-6da4-4bd0-99c1-0115e2aa4623","Type":"ContainerStarted","Data":"8e75daa2ae4acd0e378b016be29c9fcb74ae696162fee3982e6be627257d7b0f"}
Sep 08 20:26:26.733811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.733781    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/0.log"
Sep 08 20:26:26.733995 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.733824    2561 generic.go:356] "Generic (PLEG): container finished" podID="be92c86c-f31f-43e5-94a5-66d0d8dc4415" containerID="02fbd077d06753e5e28bcc11da768445ae4761a939741ce1bd0b2bb8b23bd312" exitCode=255
Sep 08 20:26:26.733995 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.733916    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-h4c88" event={"ID":"be92c86c-f31f-43e5-94a5-66d0d8dc4415","Type":"ContainerDied","Data":"02fbd077d06753e5e28bcc11da768445ae4761a939741ce1bd0b2bb8b23bd312"}
Sep 08 20:26:26.734182 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.734160    2561 scope.go:117] "RemoveContainer" containerID="02fbd077d06753e5e28bcc11da768445ae4761a939741ce1bd0b2bb8b23bd312"
Sep 08 20:26:26.735802 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.735771    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kslns" event={"ID":"7b512a7f-cf0f-44bc-8689-713e89afb601","Type":"ContainerStarted","Data":"1b6c457d862353f2f075130eff4eb8a0e06895a737b3b0300f08f905f1ef7f0e"}
Sep 08 20:26:26.735928 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.735851    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:26:26.737466 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.737444    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqs79" event={"ID":"92d5f074-5b53-43a6-b31e-5136411eba01","Type":"ContainerStarted","Data":"a96edd406dc4b08de44d6f5bf13017d54a1915d366de97fc700f041e9ce6b3a9"}
Sep 08 20:26:26.749639 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.749589    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8" podStartSLOduration=2.065075297 podStartE2EDuration="7.749567726s" totalImagesPullingTime="5.684492429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:26.749149737 +0000 UTC m=+72.914320317" watchObservedRunningTime="2026-09-08 20:26:26.749567726 +0000 UTC m=+72.914738304"
Sep 08 20:26:26.793941 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.793871    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rqs79" podStartSLOduration=65.637324331 podStartE2EDuration="1m11.793851142s" totalImagesPullingTime="6.156526811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:26.78821784 +0000 UTC m=+72.953388417" watchObservedRunningTime="2026-09-08 20:26:26.793851142 +0000 UTC m=+72.959021719"
Sep 08 20:26:26.808664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.808606    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kslns" podStartSLOduration=66.959406229 podStartE2EDuration="1m12.808584697s" totalImagesPullingTime="5.849178468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:26.807772806 +0000 UTC m=+72.972943384" watchObservedRunningTime="2026-09-08 20:26:26.808584697 +0000 UTC m=+72.973755276"
Sep 08 20:26:26.821120 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.821080    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:26.821310 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.821254    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:34.821231829 +0000 UTC m=+80.986402406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : configmap references non-existent config key: service-ca.crt
Sep 08 20:26:26.821964 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.821717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:26.822107 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.821890    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 20:26:26.822107 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.822041    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:34.822024107 +0000 UTC m=+80.987194692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : secret "router-metrics-certs-default" not found
Sep 08 20:26:26.922682 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:26.922646    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:26.922841 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.922791    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 20:26:26.922883 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:26.922854    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert podName:7d8ba035-e6a4-4e15-9a1e-79b7e6742b41 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:34.922836447 +0000 UTC m=+81.088007017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tnls4" (UID: "7d8ba035-e6a4-4e15-9a1e-79b7e6742b41") : secret "networking-console-plugin-cert" not found
Sep 08 20:26:27.123546 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.123510    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:27.123754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.123564    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:27.123754 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:27.123645    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:27.123754 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:27.123649    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:27.123754 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:27.123726    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:29.123707757 +0000 UTC m=+75.288878335 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:27.123999 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:27.123780    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:29.123766257 +0000 UTC m=+75.288936810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:27.437642 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.437612    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-psrxw_f938a669-effc-4bb0-8894-fdb5be304643/dns-node-resolver/0.log"
Sep 08 20:26:27.742277 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.742203    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:26:27.742688 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.742667    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/0.log"
Sep 08 20:26:27.742759 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.742710    2561 generic.go:356] "Generic (PLEG): container finished" podID="be92c86c-f31f-43e5-94a5-66d0d8dc4415" containerID="8b70c69102a2d93e2765162380aa6a01d522a01674e0491f1f0332b1efde7a29" exitCode=255
Sep 08 20:26:27.742845 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.742815    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-h4c88" event={"ID":"be92c86c-f31f-43e5-94a5-66d0d8dc4415","Type":"ContainerDied","Data":"8b70c69102a2d93e2765162380aa6a01d522a01674e0491f1f0332b1efde7a29"}
Sep 08 20:26:27.742910 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.742882    2561 scope.go:117] "RemoveContainer" containerID="02fbd077d06753e5e28bcc11da768445ae4761a939741ce1bd0b2bb8b23bd312"
Sep 08 20:26:27.743055 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:27.743031    2561 scope.go:117] "RemoveContainer" containerID="8b70c69102a2d93e2765162380aa6a01d522a01674e0491f1f0332b1efde7a29"
Sep 08 20:26:27.743471 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:27.743220    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-h4c88_openshift-console-operator(be92c86c-f31f-43e5-94a5-66d0d8dc4415)\"" pod="openshift-console-operator/console-operator-85b454874b-h4c88" podUID="be92c86c-f31f-43e5-94a5-66d0d8dc4415"
Sep 08 20:26:28.433335 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:28.433307    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sktlj_2ebae549-d876-40d4-a52c-025f6b21dde0/node-ca/0.log"
Sep 08 20:26:28.746881 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:28.746800    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:26:28.747251 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:28.747150    2561 scope.go:117] "RemoveContainer" containerID="8b70c69102a2d93e2765162380aa6a01d522a01674e0491f1f0332b1efde7a29"
Sep 08 20:26:28.747316 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:28.747297    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-h4c88_openshift-console-operator(be92c86c-f31f-43e5-94a5-66d0d8dc4415)\"" pod="openshift-console-operator/console-operator-85b454874b-h4c88" podUID="be92c86c-f31f-43e5-94a5-66d0d8dc4415"
Sep 08 20:26:29.142595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:29.142491    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:29.142595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:29.142555    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:29.142812 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:29.142625    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:29.142812 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:29.142660    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:29.142812 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:29.142704    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:33.14268823 +0000 UTC m=+79.307858795 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:29.142812 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:29.142719    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:33.142712814 +0000 UTC m=+79.307883368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:29.234231 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:29.234203    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j69nw_74e19476-d6be-47fc-a532-4129d15638af/serve-healthcheck-canary/0.log"
Sep 08 20:26:29.572986 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:29.572940    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:29.573172 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:29.573009    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:29.749466 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:29.749438    2561 scope.go:117] "RemoveContainer" containerID="8b70c69102a2d93e2765162380aa6a01d522a01674e0491f1f0332b1efde7a29"
Sep 08 20:26:29.749823 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:29.749588    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-h4c88_openshift-console-operator(be92c86c-f31f-43e5-94a5-66d0d8dc4415)\"" pod="openshift-console-operator/console-operator-85b454874b-h4c88" podUID="be92c86c-f31f-43e5-94a5-66d0d8dc4415"
Sep 08 20:26:33.172503 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:33.172455    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:33.173042 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:33.172519    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:33.173042 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:33.172587    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:33.173042 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:33.172649    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:33.173042 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:33.172666    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:41.172649705 +0000 UTC m=+87.337820291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:33.173042 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:33.172688    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:41.172676903 +0000 UTC m=+87.337847457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:34.886120 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:34.886074    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:34.886516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:34.886174    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:34.886516 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:34.886237    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:50.886219721 +0000 UTC m=+97.051390299 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : configmap references non-existent config key: service-ca.crt
Sep 08 20:26:34.886516 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:34.886284    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 20:26:34.886516 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:34.886349    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs podName:350750d6-ab60-46cc-9421-cdb689c07574 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:50.886338191 +0000 UTC m=+97.051508745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs") pod "router-default-5c98988766-lv6fl" (UID: "350750d6-ab60-46cc-9421-cdb689c07574") : secret "router-metrics-certs-default" not found
Sep 08 20:26:34.987259 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:34.987215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:34.987428 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:34.987358    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 20:26:34.987467 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:34.987431    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert podName:7d8ba035-e6a4-4e15-9a1e-79b7e6742b41 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:50.987416252 +0000 UTC m=+97.152586806 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tnls4" (UID: "7d8ba035-e6a4-4e15-9a1e-79b7e6742b41") : secret "networking-console-plugin-cert" not found
Sep 08 20:26:41.238026 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:41.237881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:41.238484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:41.238077    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:41.238484 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:41.238240    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:41.238484 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:41.238323    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls podName:bbcd2699-0b1b-4611-96dc-18129e3364f8 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:57.238302426 +0000 UTC m=+103.403472999 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjrd7" (UID: "bbcd2699-0b1b-4611-96dc-18129e3364f8") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:41.238656 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:41.238498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbcd2699-0b1b-4611-96dc-18129e3364f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:41.358270 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:41.358235    2561 scope.go:117] "RemoveContainer" containerID="8b70c69102a2d93e2765162380aa6a01d522a01674e0491f1f0332b1efde7a29"
Sep 08 20:26:41.783765 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:41.783733    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:26:41.783938 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:41.783806    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-h4c88" event={"ID":"be92c86c-f31f-43e5-94a5-66d0d8dc4415","Type":"ContainerStarted","Data":"bcbf23a6c0b4413b7f5022219410ebe80c2603c4b1924426a8ef8dea260f2d78"}
Sep 08 20:26:41.784124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:41.784095    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:41.799535 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:41.799474    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-h4c88" podStartSLOduration=17.043740804 podStartE2EDuration="22.799454033s" totalImagesPullingTime="5.755713229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:41.799330925 +0000 UTC m=+87.964501500" watchObservedRunningTime="2026-09-08 20:26:41.799454033 +0000 UTC m=+87.964624606"
Sep 08 20:26:42.511070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:42.511039    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-h4c88"
Sep 08 20:26:47.486323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.486285    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-bc4cl"]
Sep 08 20:26:47.491947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.491916    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-bc4cl"
Sep 08 20:26:47.494691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.494669    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-678t6\""
Sep 08 20:26:47.494835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.494673    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 08 20:26:47.494835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.494807    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 08 20:26:47.501810 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.501789    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-bc4cl"]
Sep 08 20:26:47.591938 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.591878    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h95m\" (UniqueName: \"kubernetes.io/projected/2b29a176-eaab-4b02-83fd-1334280c7b3d-kube-api-access-2h95m\") pod \"downloads-85d98c998f-bc4cl\" (UID: \"2b29a176-eaab-4b02-83fd-1334280c7b3d\") " pod="openshift-console/downloads-85d98c998f-bc4cl"
Sep 08 20:26:47.692624 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.692581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h95m\" (UniqueName: \"kubernetes.io/projected/2b29a176-eaab-4b02-83fd-1334280c7b3d-kube-api-access-2h95m\") pod \"downloads-85d98c998f-bc4cl\" (UID: \"2b29a176-eaab-4b02-83fd-1334280c7b3d\") " pod="openshift-console/downloads-85d98c998f-bc4cl"
Sep 08 20:26:47.704016 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.703981    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h95m\" (UniqueName: \"kubernetes.io/projected/2b29a176-eaab-4b02-83fd-1334280c7b3d-kube-api-access-2h95m\") pod \"downloads-85d98c998f-bc4cl\" (UID: \"2b29a176-eaab-4b02-83fd-1334280c7b3d\") " pod="openshift-console/downloads-85d98c998f-bc4cl"
Sep 08 20:26:47.800850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:47.800771    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-bc4cl"
Sep 08 20:26:48.076663 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:48.076591    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-bc4cl"]
Sep 08 20:26:48.079945 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:48.079917    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b29a176_eaab_4b02_83fd_1334280c7b3d.slice/crio-087fc3a6afc33a2a904e45e5d987217a65563b04747d96ad430111e949a5ac34 WatchSource:0}: Error finding container 087fc3a6afc33a2a904e45e5d987217a65563b04747d96ad430111e949a5ac34: Status 404 returned error can't find the container with id 087fc3a6afc33a2a904e45e5d987217a65563b04747d96ad430111e949a5ac34
Sep 08 20:26:48.808058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:48.808024    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-bc4cl" event={"ID":"2b29a176-eaab-4b02-83fd-1334280c7b3d","Type":"ContainerStarted","Data":"087fc3a6afc33a2a904e45e5d987217a65563b04747d96ad430111e949a5ac34"}
Sep 08 20:26:50.922975 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:50.922934    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:50.923390 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:50.923013    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:50.923734 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:50.923712    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350750d6-ab60-46cc-9421-cdb689c07574-service-ca-bundle\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:50.925722 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:50.925699    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350750d6-ab60-46cc-9421-cdb689c07574-metrics-certs\") pod \"router-default-5c98988766-lv6fl\" (UID: \"350750d6-ab60-46cc-9421-cdb689c07574\") " pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:50.963707 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:50.963646    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:51.024575 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.024478    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:51.027232 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.027120    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8ba035-e6a4-4e15-9a1e-79b7e6742b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tnls4\" (UID: \"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:51.058917 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.058852    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4"
Sep 08 20:26:51.242674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.242588    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c98988766-lv6fl"]
Sep 08 20:26:51.245826 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:51.245801    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350750d6_ab60_46cc_9421_cdb689c07574.slice/crio-817818f996b4ae960784698fd70b6ec81341b842f57834ee6a33d5b2111a8dfa WatchSource:0}: Error finding container 817818f996b4ae960784698fd70b6ec81341b842f57834ee6a33d5b2111a8dfa: Status 404 returned error can't find the container with id 817818f996b4ae960784698fd70b6ec81341b842f57834ee6a33d5b2111a8dfa
Sep 08 20:26:51.338641 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.338603    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-tnls4"]
Sep 08 20:26:51.341863 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:51.341833    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8ba035_e6a4_4e15_9a1e_79b7e6742b41.slice/crio-8d2c7303487a11caac007a3716361efd43c6c72a3b095a4b28f8d48a55d33a73 WatchSource:0}: Error finding container 8d2c7303487a11caac007a3716361efd43c6c72a3b095a4b28f8d48a55d33a73: Status 404 returned error can't find the container with id 8d2c7303487a11caac007a3716361efd43c6c72a3b095a4b28f8d48a55d33a73
Sep 08 20:26:51.824020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.823983    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c98988766-lv6fl" event={"ID":"350750d6-ab60-46cc-9421-cdb689c07574","Type":"ContainerStarted","Data":"5364d13d868e4141ac1de29a9f5d3d3c01465c3b2d19a7989c56d4f00e052892"}
Sep 08 20:26:51.824202 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.824060    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c98988766-lv6fl" event={"ID":"350750d6-ab60-46cc-9421-cdb689c07574","Type":"ContainerStarted","Data":"817818f996b4ae960784698fd70b6ec81341b842f57834ee6a33d5b2111a8dfa"}
Sep 08 20:26:51.825168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.825144    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4" event={"ID":"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41","Type":"ContainerStarted","Data":"8d2c7303487a11caac007a3716361efd43c6c72a3b095a4b28f8d48a55d33a73"}
Sep 08 20:26:51.844585 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.844529    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5c98988766-lv6fl" podStartSLOduration=32.844513325 podStartE2EDuration="32.844513325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:51.843617132 +0000 UTC m=+98.008787710" watchObservedRunningTime="2026-09-08 20:26:51.844513325 +0000 UTC m=+98.009683900"
Sep 08 20:26:51.964672 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.964628    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:51.967650 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:51.967625    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:52.642009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.641971    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:26:52.642190 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.642043    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:26:52.644662 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.644636    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68bd9156-10b5-407c-8ef9-3ce4d0e0204b-metrics-tls\") pod \"dns-default-xmz24\" (UID: \"68bd9156-10b5-407c-8ef9-3ce4d0e0204b\") " pod="openshift-dns/dns-default-xmz24"
Sep 08 20:26:52.644909 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.644876    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"image-registry-7c6674f4b8-n8ntr\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") " pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:26:52.818960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.818929    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c65d89f69-v8qrl"]
Sep 08 20:26:52.834513 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.834474    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c65d89f69-v8qrl"]
Sep 08 20:26:52.834679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.834525    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:52.834679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.834588    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5c98988766-lv6fl"
Sep 08 20:26:52.834679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.834649    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:52.837724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.837695    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 20:26:52.837867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.837740    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 20:26:52.837867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.837745    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 20:26:52.837867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.837698    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 20:26:52.838485 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.838342    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 20:26:52.838485 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.838367    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zlxsw\""
Sep 08 20:26:52.892232 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.892159    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nwnm8\""
Sep 08 20:26:52.900260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.900232    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:26:52.906694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.906669    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vbwpr\""
Sep 08 20:26:52.914676 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.914644    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xmz24"
Sep 08 20:26:52.944567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.944529    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-serving-cert\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:52.944706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.944604    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-oauth-config\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:52.944706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.944627    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-config\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:52.944706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.944657    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-oauth-serving-cert\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:52.944706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.944684    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-service-ca\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:52.944920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:52.944729    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjpl\" (UniqueName: \"kubernetes.io/projected/dcc7a745-bf03-4fe7-9749-8cb34707ce24-kube-api-access-vnjpl\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.045251 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.045220    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-serving-cert\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.045707 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.045291    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-oauth-config\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.045707 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.045321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-config\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.045707 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.045348    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-oauth-serving-cert\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.045707 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.045382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-service-ca\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.045707 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.045431    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjpl\" (UniqueName: \"kubernetes.io/projected/dcc7a745-bf03-4fe7-9749-8cb34707ce24-kube-api-access-vnjpl\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.046496 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.046080    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-config\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.046496 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.046319    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-oauth-serving-cert\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.046496 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.046337    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-service-ca\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.048365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.048312    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-serving-cert\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.049230 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.049204    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-oauth-config\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.054660 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.054241    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjpl\" (UniqueName: \"kubernetes.io/projected/dcc7a745-bf03-4fe7-9749-8cb34707ce24-kube-api-access-vnjpl\") pod \"console-5c65d89f69-v8qrl\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") " pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.146682 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.146596    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:26:53.267845 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:53.267811    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68bd9156_10b5_407c_8ef9_3ce4d0e0204b.slice/crio-771d3a081c99c6e3825142593fa4eb1e231f7740478fe4f92c11f38214d28d67 WatchSource:0}: Error finding container 771d3a081c99c6e3825142593fa4eb1e231f7740478fe4f92c11f38214d28d67: Status 404 returned error can't find the container with id 771d3a081c99c6e3825142593fa4eb1e231f7740478fe4f92c11f38214d28d67
Sep 08 20:26:53.274535 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.274497    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xmz24"]
Sep 08 20:26:53.275577 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.275553    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c6674f4b8-n8ntr"]
Sep 08 20:26:53.275728 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:53.275647    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4283e3_b5ae_4809_8bee_8a0320e57de4.slice/crio-048169f0c4402895f8bd5b524b70a1e9455c61883970c4d2cf6c43d9a875851e WatchSource:0}: Error finding container 048169f0c4402895f8bd5b524b70a1e9455c61883970c4d2cf6c43d9a875851e: Status 404 returned error can't find the container with id 048169f0c4402895f8bd5b524b70a1e9455c61883970c4d2cf6c43d9a875851e
Sep 08 20:26:53.449685 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.449652    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c65d89f69-v8qrl"]
Sep 08 20:26:53.453324 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:53.453293    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc7a745_bf03_4fe7_9749_8cb34707ce24.slice/crio-485d41d05a3712bbd36faea68d4e1c26d0aca41cbeeda913526f6c8d88d5609a WatchSource:0}: Error finding container 485d41d05a3712bbd36faea68d4e1c26d0aca41cbeeda913526f6c8d88d5609a: Status 404 returned error can't find the container with id 485d41d05a3712bbd36faea68d4e1c26d0aca41cbeeda913526f6c8d88d5609a
Sep 08 20:26:53.834116 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.834030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xmz24" event={"ID":"68bd9156-10b5-407c-8ef9-3ce4d0e0204b","Type":"ContainerStarted","Data":"771d3a081c99c6e3825142593fa4eb1e231f7740478fe4f92c11f38214d28d67"}
Sep 08 20:26:53.835912 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.835832    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr" event={"ID":"cb4283e3-b5ae-4809-8bee-8a0320e57de4","Type":"ContainerStarted","Data":"df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa"}
Sep 08 20:26:53.835912 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.835886    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr" event={"ID":"cb4283e3-b5ae-4809-8bee-8a0320e57de4","Type":"ContainerStarted","Data":"048169f0c4402895f8bd5b524b70a1e9455c61883970c4d2cf6c43d9a875851e"}
Sep 08 20:26:53.836100 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.835939    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:26:53.837635 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.837597    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4" event={"ID":"7d8ba035-e6a4-4e15-9a1e-79b7e6742b41","Type":"ContainerStarted","Data":"986eec3664c40af52f6256961160781f5611bd517fd65bc4f1457c589ab97c5e"}
Sep 08 20:26:53.838996 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.838968    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c65d89f69-v8qrl" event={"ID":"dcc7a745-bf03-4fe7-9749-8cb34707ce24","Type":"ContainerStarted","Data":"485d41d05a3712bbd36faea68d4e1c26d0aca41cbeeda913526f6c8d88d5609a"}
Sep 08 20:26:53.857795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.857739    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr" podStartSLOduration=78.857724205 podStartE2EDuration="1m18.857724205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:53.857374877 +0000 UTC m=+100.022545446" watchObservedRunningTime="2026-09-08 20:26:53.857724205 +0000 UTC m=+100.022894781"
Sep 08 20:26:53.879398 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:53.879341    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-tnls4" podStartSLOduration=33.246246735 podStartE2EDuration="34.879322742s" totalImagesPullingTime="1.633076007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:53.878201386 +0000 UTC m=+100.043371963" watchObservedRunningTime="2026-09-08 20:26:53.879322742 +0000 UTC m=+100.044493317"
Sep 08 20:26:57.290639 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.290580    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:57.293109 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.293078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbcd2699-0b1b-4611-96dc-18129e3364f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjrd7\" (UID: \"bbcd2699-0b1b-4611-96dc-18129e3364f8\") " pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:57.300083 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.300035    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kjrd7"
Sep 08 20:26:57.677103 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.677068    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kjrd7"]
Sep 08 20:26:57.680606 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:26:57.680581    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbcd2699_0b1b_4611_96dc_18129e3364f8.slice/crio-bbbec6b2cc4a98d578575c65c842486490b8af42f6a11e30a1bf6350c9c121fd WatchSource:0}: Error finding container bbbec6b2cc4a98d578575c65c842486490b8af42f6a11e30a1bf6350c9c121fd: Status 404 returned error can't find the container with id bbbec6b2cc4a98d578575c65c842486490b8af42f6a11e30a1bf6350c9c121fd
Sep 08 20:26:57.747220 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.746796    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kslns"
Sep 08 20:26:57.856866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.856777    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c65d89f69-v8qrl" event={"ID":"dcc7a745-bf03-4fe7-9749-8cb34707ce24","Type":"ContainerStarted","Data":"9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3"}
Sep 08 20:26:57.858704 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.858679    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xmz24" event={"ID":"68bd9156-10b5-407c-8ef9-3ce4d0e0204b","Type":"ContainerStarted","Data":"cde9033485ea37999c88ea780b2cfe64f0e71a0ca78b8c4bde549870a25139b8"}
Sep 08 20:26:57.858830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.858727    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xmz24" event={"ID":"68bd9156-10b5-407c-8ef9-3ce4d0e0204b","Type":"ContainerStarted","Data":"ac744afb4aae2ed1f11624aadcd051eb11b2631412d9c7f0c51eed31f5c094de"}
Sep 08 20:26:57.858830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.858795    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xmz24"
Sep 08 20:26:57.860367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.860343    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kjrd7" event={"ID":"bbcd2699-0b1b-4611-96dc-18129e3364f8","Type":"ContainerStarted","Data":"101f2e40401541b3062258f704d8cddf09a6dcadde75e51bcbcb01c067062db4"}
Sep 08 20:26:57.860461 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.860396    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kjrd7" event={"ID":"bbcd2699-0b1b-4611-96dc-18129e3364f8","Type":"ContainerStarted","Data":"bbbec6b2cc4a98d578575c65c842486490b8af42f6a11e30a1bf6350c9c121fd"}
Sep 08 20:26:57.874335 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.874270    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c65d89f69-v8qrl" podStartSLOduration=1.9231791679999999 podStartE2EDuration="5.87425148s" totalImagesPullingTime="3.951072312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:57.873950403 +0000 UTC m=+104.039120982" watchObservedRunningTime="2026-09-08 20:26:57.87425148 +0000 UTC m=+104.039422058"
Sep 08 20:26:57.890485 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:57.890435    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xmz24" podStartSLOduration=65.759591671 podStartE2EDuration="1m9.890420249s" totalImagesPullingTime="4.130828578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:57.889712499 +0000 UTC m=+104.054883077" watchObservedRunningTime="2026-09-08 20:26:57.890420249 +0000 UTC m=+104.055590825"
Sep 08 20:26:58.729054 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.729014    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-4d542"]
Sep 08 20:26:58.731675 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.731645    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:58.735412 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.735385    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 20:26:58.735547 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.735384    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 20:26:58.735547 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.735438    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 20:26:58.735626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.735568    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 08 20:26:58.735626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.735577    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 08 20:26:58.735626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.735607    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-d7b9z\""
Sep 08 20:26:58.745904 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.745866    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-4d542"]
Sep 08 20:26:58.904777 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.904735    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:58.905003 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.904794    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfmg2\" (UniqueName: \"kubernetes.io/projected/48d57a6b-d406-4864-924d-bea5515378f1-kube-api-access-dfmg2\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:58.905003 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.904917    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48d57a6b-d406-4864-924d-bea5515378f1-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:58.905003 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:58.904961    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.006272 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.006182    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48d57a6b-d406-4864-924d-bea5515378f1-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.006469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.006300    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.006805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.006557    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.006805 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:59.006622    2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 08 20:26:59.006805 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:26:59.006691    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-tls podName:48d57a6b-d406-4864-924d-bea5515378f1 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:59.506673807 +0000 UTC m=+105.671844361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-4d542" (UID: "48d57a6b-d406-4864-924d-bea5515378f1") : secret "prometheus-operator-tls" not found
Sep 08 20:26:59.006805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.006624    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfmg2\" (UniqueName: \"kubernetes.io/projected/48d57a6b-d406-4864-924d-bea5515378f1-kube-api-access-dfmg2\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.007164 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.007143    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48d57a6b-d406-4864-924d-bea5515378f1-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.009189 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.009167    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.017323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.017292    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfmg2\" (UniqueName: \"kubernetes.io/projected/48d57a6b-d406-4864-924d-bea5515378f1-kube-api-access-dfmg2\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.512438 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.512397    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.515087 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.515062    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48d57a6b-d406-4864-924d-bea5515378f1-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-4d542\" (UID: \"48d57a6b-d406-4864-924d-bea5515378f1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:26:59.642927 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:26:59.642879    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542"
Sep 08 20:27:03.146950 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:03.146911    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:27:03.147452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:03.146984    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:27:03.152463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:03.152437    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:27:03.886970 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:03.886936    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:27:05.021011 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:05.020939    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-4d542"]
Sep 08 20:27:05.024602 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:27:05.024569    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d57a6b_d406_4864_924d_bea5515378f1.slice/crio-554b25a0ccf56b2ba889697ef45a9ccea16b61350f22bcad877f2ef412aa45ca WatchSource:0}: Error finding container 554b25a0ccf56b2ba889697ef45a9ccea16b61350f22bcad877f2ef412aa45ca: Status 404 returned error can't find the container with id 554b25a0ccf56b2ba889697ef45a9ccea16b61350f22bcad877f2ef412aa45ca
Sep 08 20:27:05.894034 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:05.893990    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kjrd7" event={"ID":"bbcd2699-0b1b-4611-96dc-18129e3364f8","Type":"ContainerStarted","Data":"39fec437c9139cd224941b1c5c2d650ea3b6270299045db032b121050dda793c"}
Sep 08 20:27:05.896752 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:05.895678    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542" event={"ID":"48d57a6b-d406-4864-924d-bea5515378f1","Type":"ContainerStarted","Data":"554b25a0ccf56b2ba889697ef45a9ccea16b61350f22bcad877f2ef412aa45ca"}
Sep 08 20:27:05.900617 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:05.900586    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-bc4cl" event={"ID":"2b29a176-eaab-4b02-83fd-1334280c7b3d","Type":"ContainerStarted","Data":"f3b01bc233092eca0f7ba3d70a592d41d1afddb3beebdba8bb860fab13e41509"}
Sep 08 20:27:05.900880 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:05.900832    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-bc4cl"
Sep 08 20:27:05.914369 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:05.914294    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-bc4cl"
Sep 08 20:27:05.923029 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:05.922262    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-bc4cl" podStartSLOduration=2.152476608 podStartE2EDuration="18.922237795s" totalImagesPullingTime="16.769761187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:05.921210677 +0000 UTC m=+112.086381254" watchObservedRunningTime="2026-09-08 20:27:05.922237795 +0000 UTC m=+112.087408372"
Sep 08 20:27:07.866552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:07.866516    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xmz24"
Sep 08 20:27:07.912718 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:07.912683    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kjrd7" event={"ID":"bbcd2699-0b1b-4611-96dc-18129e3364f8","Type":"ContainerStarted","Data":"abec06108b58fa6642f88b2fb53404445445824cf6ae2401ea33f37ad326709f"}
Sep 08 20:27:07.915600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:07.915566    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542" event={"ID":"48d57a6b-d406-4864-924d-bea5515378f1","Type":"ContainerStarted","Data":"30ac433fd2ebf5146c76b6ed550dfe4ba650ae81096aba52e0ee3665e96cb734"}
Sep 08 20:27:07.915765 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:07.915627    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542" event={"ID":"48d57a6b-d406-4864-924d-bea5515378f1","Type":"ContainerStarted","Data":"aa2e6b214df85c6dd3e251fec4d8a0d4a18272bd94b600712da5efa8a31ddd2c"}
Sep 08 20:27:07.933722 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:07.933669    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kjrd7" podStartSLOduration=33.286472238 podStartE2EDuration="42.933650783s" totalImagesPullingTime="9.647178545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:26:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:07.932105426 +0000 UTC m=+114.097276002" watchObservedRunningTime="2026-09-08 20:27:07.933650783 +0000 UTC m=+114.098821385"
Sep 08 20:27:07.949089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:07.949024    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-4d542" podStartSLOduration=7.497589101 podStartE2EDuration="9.949000372s" totalImagesPullingTime="2.451411271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:07.948577145 +0000 UTC m=+114.113747735" watchObservedRunningTime="2026-09-08 20:27:07.949000372 +0000 UTC m=+114.114170949"
Sep 08 20:27:10.212926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.212875    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"]
Sep 08 20:27:10.262538 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.262501    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"]
Sep 08 20:27:10.262724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.262659    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.266011 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.265980    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 08 20:27:10.266168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.265980    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 08 20:27:10.266168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.265992    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8cgjc\""
Sep 08 20:27:10.287034 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.286999    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"]
Sep 08 20:27:10.287226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.287064    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vmw6j"]
Sep 08 20:27:10.287226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.287156    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.289869 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.289827    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9q65p\""
Sep 08 20:27:10.290042 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.289882    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 08 20:27:10.290121 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.290074    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 08 20:27:10.290181 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.290145    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 08 20:27:10.310013 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.309980    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.310192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310020    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.310192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310066    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.310192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.310192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310126    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p55dh\" (UniqueName: \"kubernetes.io/projected/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-api-access-p55dh\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.310414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310231    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.310414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310293    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.310414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310321    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngzhk\" (UniqueName: \"kubernetes.io/projected/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-kube-api-access-ngzhk\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.310414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.310567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.310470    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.315333 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.315303    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"]
Sep 08 20:27:10.315470 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.315447    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.318238 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.318215    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 20:27:10.318430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.318397    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nwv9r\""
Sep 08 20:27:10.318430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.318424    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 20:27:10.318609 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.318557    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 20:27:10.333780 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.333749    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c6674f4b8-n8ntr"]
Sep 08 20:27:10.339690 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.339666    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:27:10.411246 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411207    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.411438 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411260    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.411438 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-wtmp\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.411566 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411462    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-root\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.411566 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p55dh\" (UniqueName: \"kubernetes.io/projected/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-api-access-p55dh\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.411566 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411545    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-sys\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.411715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411573    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.411715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411692    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.411822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411743    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.411822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411767    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngzhk\" (UniqueName: \"kubernetes.io/projected/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-kube-api-access-ngzhk\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.411822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411791    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-tls\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.412027 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6b0599e-1af2-48a8-9171-1777ac9074b2-metrics-client-ca\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.412027 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411950    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.412027 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.411980    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.412027 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szb9h\" (UniqueName: \"kubernetes.io/projected/b6b0599e-1af2-48a8-9171-1777ac9074b2-kube-api-access-szb9h\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.412245 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412049    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-accelerators-collector-config\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.412245 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412091    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.412245 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412100    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.412245 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412119    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.412245 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412150    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-textfile\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.412245 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412114    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.412524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412419    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.412524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.412487    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.414801 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.414773    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.415230 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.415186    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.415719 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.415696    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.416161 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.416137    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.420396 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.420364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngzhk\" (UniqueName: \"kubernetes.io/projected/e56dd66b-f18a-4d3c-a71a-b06ae1204c7a-kube-api-access-ngzhk\") pod \"openshift-state-metrics-657fd6c54-7xffg\" (UID: \"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.420525 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.420492    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p55dh\" (UniqueName: \"kubernetes.io/projected/a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07-kube-api-access-p55dh\") pod \"kube-state-metrics-74cb57df7-cdmjn\" (UID: \"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.513342 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513249    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-sys\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513342 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513297    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513566 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-sys\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513625 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-tls\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6b0599e-1af2-48a8-9171-1777ac9074b2-metrics-client-ca\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513666    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szb9h\" (UniqueName: \"kubernetes.io/projected/b6b0599e-1af2-48a8-9171-1777ac9074b2-kube-api-access-szb9h\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513772 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513698    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-accelerators-collector-config\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513772 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513747    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-textfile\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513797    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-wtmp\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.513873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513825    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-root\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.514041 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.513917    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-root\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.514413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.514389    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6b0599e-1af2-48a8-9171-1777ac9074b2-metrics-client-ca\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.514625 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.514525    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-textfile\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.514733 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.514645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-wtmp\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.515419 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.515365    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-accelerators-collector-config\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.517017 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.516962    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.517136 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.517082    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6b0599e-1af2-48a8-9171-1777ac9074b2-node-exporter-tls\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.522066 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.522021    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szb9h\" (UniqueName: \"kubernetes.io/projected/b6b0599e-1af2-48a8-9171-1777ac9074b2-kube-api-access-szb9h\") pod \"node-exporter-vmw6j\" (UID: \"b6b0599e-1af2-48a8-9171-1777ac9074b2\") " pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.573943 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.573879    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"
Sep 08 20:27:10.597801 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.597757    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"
Sep 08 20:27:10.626922 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.626862    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vmw6j"
Sep 08 20:27:10.884885 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.884859    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg"]
Sep 08 20:27:10.887639 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:27:10.887605    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode56dd66b_f18a_4d3c_a71a_b06ae1204c7a.slice/crio-020b26a1fa10b11944a9bd6674a930da465a7404f6c66ffc34d4e8669e397e96 WatchSource:0}: Error finding container 020b26a1fa10b11944a9bd6674a930da465a7404f6c66ffc34d4e8669e397e96: Status 404 returned error can't find the container with id 020b26a1fa10b11944a9bd6674a930da465a7404f6c66ffc34d4e8669e397e96
Sep 08 20:27:10.918242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.918211    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn"]
Sep 08 20:27:10.923139 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:27:10.923109    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80cc9ba_7fec_4b0d_b5cd_2b4cec6acc07.slice/crio-6645f5fe26a6f58027e4768be81be2cf8ee20f2b932b8bc10b64eccbb818217d WatchSource:0}: Error finding container 6645f5fe26a6f58027e4768be81be2cf8ee20f2b932b8bc10b64eccbb818217d: Status 404 returned error can't find the container with id 6645f5fe26a6f58027e4768be81be2cf8ee20f2b932b8bc10b64eccbb818217d
Sep 08 20:27:10.929968 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.929934    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vmw6j" event={"ID":"b6b0599e-1af2-48a8-9171-1777ac9074b2","Type":"ContainerStarted","Data":"d5b7fd02d828e510351ab7311c67da2c77af5830a2cd7b70e2ea0ba58c202544"}
Sep 08 20:27:10.931283 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:10.931251    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg" event={"ID":"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a","Type":"ContainerStarted","Data":"020b26a1fa10b11944a9bd6674a930da465a7404f6c66ffc34d4e8669e397e96"}
Sep 08 20:27:11.155496 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.155460    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 20:27:11.180239 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.180199    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 20:27:11.180453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.180432    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.184713    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.184985    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.185187    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.185397    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.185643    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.185692    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.185828    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.185888    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 08 20:27:11.186211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.186164    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ww5ck\""
Sep 08 20:27:11.186754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.186342    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 08 20:27:11.222737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222528    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.222737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222581    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-web-config\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.222737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222611    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.222737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222669    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222751    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/badbccaa-859d-4250-b6cc-64f9997152e2-config-out\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222798    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94wwx\" (UniqueName: \"kubernetes.io/projected/badbccaa-859d-4250-b6cc-64f9997152e2-kube-api-access-94wwx\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222844    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/badbccaa-859d-4250-b6cc-64f9997152e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222904    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222943    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.222969    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.223015    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.223080    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.223354 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.223129    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324399 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324360    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324428    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324454    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-web-config\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324477    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324506    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324539    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/badbccaa-859d-4250-b6cc-64f9997152e2-config-out\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324573    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94wwx\" (UniqueName: \"kubernetes.io/projected/badbccaa-859d-4250-b6cc-64f9997152e2-kube-api-access-94wwx\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324611    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/badbccaa-859d-4250-b6cc-64f9997152e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324638    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324684    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324728    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.324884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.324759    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.325588 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.325545    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.326130 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:27:11.326061    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-trusted-ca-bundle podName:badbccaa-859d-4250-b6cc-64f9997152e2 nodeName:}" failed. No retries permitted until 2026-09-08 20:27:11.826036155 +0000 UTC m=+117.991206724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "badbccaa-859d-4250-b6cc-64f9997152e2") : configmap references non-existent config key: ca-bundle.crt
Sep 08 20:27:11.326927 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.326641    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.328422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.328067    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.328517 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.328452    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.328517 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.328487    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.329104 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.329083    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.329336 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.329275    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.329816 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.329791    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/badbccaa-859d-4250-b6cc-64f9997152e2-config-out\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.330537 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.330518    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/badbccaa-859d-4250-b6cc-64f9997152e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.331012 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.330990    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-web-config\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.331341 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.331314    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/badbccaa-859d-4250-b6cc-64f9997152e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.334694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.334671    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94wwx\" (UniqueName: \"kubernetes.io/projected/badbccaa-859d-4250-b6cc-64f9997152e2-kube-api-access-94wwx\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.832460 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.832410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.833554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.833498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badbccaa-859d-4250-b6cc-64f9997152e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"badbccaa-859d-4250-b6cc-64f9997152e2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:11.939615 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.939349    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn" event={"ID":"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07","Type":"ContainerStarted","Data":"6645f5fe26a6f58027e4768be81be2cf8ee20f2b932b8bc10b64eccbb818217d"}
Sep 08 20:27:11.942378 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.942346    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg" event={"ID":"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a","Type":"ContainerStarted","Data":"3339b3b952765eda30ef1d79fa272fdfabe6b6c5f5645dd5fb51cb0170a02f67"}
Sep 08 20:27:11.942510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:11.942403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg" event={"ID":"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a","Type":"ContainerStarted","Data":"7a65dc03ee167bac5c83325e78ebb66d33803aa65780c7c02160893446579478"}
Sep 08 20:27:12.095391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:12.095276    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 20:27:12.439508 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:12.439172    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 20:27:12.450552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:12.450511    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c65d89f69-v8qrl"]
Sep 08 20:27:12.586174 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:27:12.586131    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbadbccaa_859d_4250_b6cc_64f9997152e2.slice/crio-33a0667c690c3b415d09f4c3f3f56184a0174411fbed0085352328b2ab7a3c3b WatchSource:0}: Error finding container 33a0667c690c3b415d09f4c3f3f56184a0174411fbed0085352328b2ab7a3c3b: Status 404 returned error can't find the container with id 33a0667c690c3b415d09f4c3f3f56184a0174411fbed0085352328b2ab7a3c3b
Sep 08 20:27:12.947932 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:12.947868    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"badbccaa-859d-4250-b6cc-64f9997152e2","Type":"ContainerStarted","Data":"33a0667c690c3b415d09f4c3f3f56184a0174411fbed0085352328b2ab7a3c3b"}
Sep 08 20:27:14.958248 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.958144    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn" event={"ID":"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07","Type":"ContainerStarted","Data":"2c081b4d6990c27e7d058c323e13f06791388bb4b9dbce17e725d9e0aa83ebdb"}
Sep 08 20:27:14.958704 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.958251    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn" event={"ID":"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07","Type":"ContainerStarted","Data":"01f77efc2f305c927c2efc26f8fd94450e7570ff7b68857f5cd551dfea07b440"}
Sep 08 20:27:14.958704 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.958279    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn" event={"ID":"a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07","Type":"ContainerStarted","Data":"2cb26585cb9c0474472b78c54bc149e45a0767bf69bedd1bc3320f6872a9e9de"}
Sep 08 20:27:14.959850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.959823    2561 generic.go:356] "Generic (PLEG): container finished" podID="badbccaa-859d-4250-b6cc-64f9997152e2" containerID="22bafdb60e867b109ad040546092bc700ad2b4f4750d73c42c80070bf6363732" exitCode=0
Sep 08 20:27:14.959988 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.959884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"badbccaa-859d-4250-b6cc-64f9997152e2","Type":"ContainerDied","Data":"22bafdb60e867b109ad040546092bc700ad2b4f4750d73c42c80070bf6363732"}
Sep 08 20:27:14.962364 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.962338    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg" event={"ID":"e56dd66b-f18a-4d3c-a71a-b06ae1204c7a","Type":"ContainerStarted","Data":"e6d95ee1a3e9ef774424327c3e29471d9ad2f04eb2670abe1f9e7a47b6f78652"}
Sep 08 20:27:14.964091 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.964070    2561 generic.go:356] "Generic (PLEG): container finished" podID="b6b0599e-1af2-48a8-9171-1777ac9074b2" containerID="9f9aa5efd5fe5aa462c26dde30b8f97e1e28a11aaecf9d2ad55720f2ed2c7d6f" exitCode=0
Sep 08 20:27:14.964183 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.964140    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vmw6j" event={"ID":"b6b0599e-1af2-48a8-9171-1777ac9074b2","Type":"ContainerDied","Data":"9f9aa5efd5fe5aa462c26dde30b8f97e1e28a11aaecf9d2ad55720f2ed2c7d6f"}
Sep 08 20:27:14.982295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.982235    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-cdmjn" podStartSLOduration=2.071529867 podStartE2EDuration="4.982215047s" totalImagesPullingTime="2.91068518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:14.980372302 +0000 UTC m=+121.145542902" watchObservedRunningTime="2026-09-08 20:27:14.982215047 +0000 UTC m=+121.147385625"
Sep 08 20:27:14.997327 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:14.997277    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-7xffg" podStartSLOduration=2.350417578 podStartE2EDuration="4.997260472s" totalImagesPullingTime="2.646842894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:14.995926776 +0000 UTC m=+121.161097357" watchObservedRunningTime="2026-09-08 20:27:14.997260472 +0000 UTC m=+121.162431042"
Sep 08 20:27:15.970509 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:15.970474    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vmw6j" event={"ID":"b6b0599e-1af2-48a8-9171-1777ac9074b2","Type":"ContainerStarted","Data":"7a59e2ffcfd9e75c0331386af091573f848bb076baaf4517edff6270b1e3b24a"}
Sep 08 20:27:15.971066 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:15.970537    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vmw6j" event={"ID":"b6b0599e-1af2-48a8-9171-1777ac9074b2","Type":"ContainerStarted","Data":"4019ba01d9251eb951becbef7f39a1681f8e086d5d98fc3900b67c40f61cbc27"}
Sep 08 20:27:15.990876 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:15.990822    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vmw6j" podStartSLOduration=2.741173994 podStartE2EDuration="5.990805785s" totalImagesPullingTime="3.14969001s" totalInitContainerRuntime="99.941781ms" isStatefulPod=true podCreationTimestamp="2026-09-08 20:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:15.989790753 +0000 UTC m=+122.154961363" watchObservedRunningTime="2026-09-08 20:27:15.990805785 +0000 UTC m=+122.155976361"
Sep 08 20:27:17.982835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:17.982767    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"badbccaa-859d-4250-b6cc-64f9997152e2","Type":"ContainerStarted","Data":"2d34ff2a987b5f2a977e015b88ddf97569256ae016d52b135ea0d1fe272a24b9"}
Sep 08 20:27:17.983204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:17.982864    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"badbccaa-859d-4250-b6cc-64f9997152e2","Type":"ContainerStarted","Data":"71c4ad75f1eea447789ddb7b9b09063b99093982b341cb476a24d50b868ee0f0"}
Sep 08 20:27:17.983204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:17.982910    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"badbccaa-859d-4250-b6cc-64f9997152e2","Type":"ContainerStarted","Data":"68b1af89e2465b5457f5f180293bb0b7d5365c6853eb011e94a2011f592bfa9c"}
Sep 08 20:27:17.983204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:17.982945    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"badbccaa-859d-4250-b6cc-64f9997152e2","Type":"ContainerStarted","Data":"c8d0378d01c04258ea30f51f29c401e883d63381328ca8980ec71aa603916e4f"}
Sep 08 20:27:18.990577 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:18.990539    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"badbccaa-859d-4250-b6cc-64f9997152e2","Type":"ContainerStarted","Data":"e2537595430b00d1efe0f4ad72a90b95f1c2d5aee366e91a34041578ccea488a"}
Sep 08 20:27:19.997819 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:19.997773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"badbccaa-859d-4250-b6cc-64f9997152e2","Type":"ContainerStarted","Data":"7aab717e1733dcef1a5b990c51d045fee7e112050306fd0c9be28e80f4b20de5"}
Sep 08 20:27:20.025877 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:20.025822    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.132640125 podStartE2EDuration="9.025804705s" totalImagesPullingTime="5.873875271s" totalInitContainerRuntime="19.289309ms" isStatefulPod=false podCreationTimestamp="2026-09-08 20:27:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:20.024930877 +0000 UTC m=+126.190101453" watchObservedRunningTime="2026-09-08 20:27:20.025804705 +0000 UTC m=+126.190975281"
Sep 08 20:27:23.143813 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:23.143754    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:27:23.146302 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:23.146264    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f559af35-ddd5-4034-93fb-94b0e072bbd0-metrics-certs\") pod \"network-metrics-daemon-qxhpg\" (UID: \"f559af35-ddd5-4034-93fb-94b0e072bbd0\") " pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:27:23.383797 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:23.383764    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kbhtv\""
Sep 08 20:27:23.392272 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:23.392240    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qxhpg"
Sep 08 20:27:23.702914 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:23.702823    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qxhpg"]
Sep 08 20:27:23.705866 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:27:23.705825    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf559af35_ddd5_4034_93fb_94b0e072bbd0.slice/crio-d9362c5023f7f2aeef75c648867958dbf794be980e142a8b1242c0604fbe7fcb WatchSource:0}: Error finding container d9362c5023f7f2aeef75c648867958dbf794be980e142a8b1242c0604fbe7fcb: Status 404 returned error can't find the container with id d9362c5023f7f2aeef75c648867958dbf794be980e142a8b1242c0604fbe7fcb
Sep 08 20:27:24.014515 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:24.014434    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qxhpg" event={"ID":"f559af35-ddd5-4034-93fb-94b0e072bbd0","Type":"ContainerStarted","Data":"d9362c5023f7f2aeef75c648867958dbf794be980e142a8b1242c0604fbe7fcb"}
Sep 08 20:27:26.024156 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:26.024122    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qxhpg" event={"ID":"f559af35-ddd5-4034-93fb-94b0e072bbd0","Type":"ContainerStarted","Data":"1b724fb40bcb9a7fc6863af0dc72c0e417135db1055e91ffc15e5906bfd6a95b"}
Sep 08 20:27:26.024673 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:26.024176    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qxhpg" event={"ID":"f559af35-ddd5-4034-93fb-94b0e072bbd0","Type":"ContainerStarted","Data":"301204460fd384165d32512bbe057d7677b6759f79f1bd7bf8b860500d0af8ab"}
Sep 08 20:27:26.044804 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:26.044749    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qxhpg" podStartSLOduration=130.620156091 podStartE2EDuration="2m12.044731504s" totalImagesPullingTime="1.424575413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:26.042930342 +0000 UTC m=+132.208100920" watchObservedRunningTime="2026-09-08 20:27:26.044731504 +0000 UTC m=+132.209902081"
Sep 08 20:27:35.358994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.358925    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr" podUID="cb4283e3-b5ae-4809-8bee-8a0320e57de4" containerName="registry" containerID="cri-o://df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa" gracePeriod=30
Sep 08 20:27:35.627036 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.626965    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:27:35.746441 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746406    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-installation-pull-secrets\") pod \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") "
Sep 08 20:27:35.746636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746456    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bwsk\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-kube-api-access-7bwsk\") pod \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") "
Sep 08 20:27:35.746636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746479    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-certificates\") pod \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") "
Sep 08 20:27:35.746636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746500    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb4283e3-b5ae-4809-8bee-8a0320e57de4-ca-trust-extracted\") pod \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") "
Sep 08 20:27:35.746636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746519    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-trusted-ca\") pod \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") "
Sep 08 20:27:35.746636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746573    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") pod \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") "
Sep 08 20:27:35.746636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746602    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-bound-sa-token\") pod \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") "
Sep 08 20:27:35.746990 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746659    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-image-registry-private-configuration\") pod \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\" (UID: \"cb4283e3-b5ae-4809-8bee-8a0320e57de4\") "
Sep 08 20:27:35.746990 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.746947    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cb4283e3-b5ae-4809-8bee-8a0320e57de4" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:35.747084 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.747030    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cb4283e3-b5ae-4809-8bee-8a0320e57de4" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:35.749156 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.749117    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cb4283e3-b5ae-4809-8bee-8a0320e57de4" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:27:35.749265 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.749184    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cb4283e3-b5ae-4809-8bee-8a0320e57de4" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:27:35.749265 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.749242    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cb4283e3-b5ae-4809-8bee-8a0320e57de4" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:27:35.749460 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.749432    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cb4283e3-b5ae-4809-8bee-8a0320e57de4" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:27:35.749536 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.749515    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-kube-api-access-7bwsk" (OuterVolumeSpecName: "kube-api-access-7bwsk") pod "cb4283e3-b5ae-4809-8bee-8a0320e57de4" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4"). InnerVolumeSpecName "kube-api-access-7bwsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:27:35.757162 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.757135    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb4283e3-b5ae-4809-8bee-8a0320e57de4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cb4283e3-b5ae-4809-8bee-8a0320e57de4" (UID: "cb4283e3-b5ae-4809-8bee-8a0320e57de4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:27:35.848167 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.848131    2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-certificates\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:35.848167 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.848163    2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb4283e3-b5ae-4809-8bee-8a0320e57de4-ca-trust-extracted\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:35.848167 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.848175    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb4283e3-b5ae-4809-8bee-8a0320e57de4-trusted-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:35.848391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.848184    2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-registry-tls\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:35.848391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.848193    2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-bound-sa-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:35.848391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.848201    2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-image-registry-private-configuration\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:35.848391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.848211    2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb4283e3-b5ae-4809-8bee-8a0320e57de4-installation-pull-secrets\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:35.848391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:35.848219    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bwsk\" (UniqueName: \"kubernetes.io/projected/cb4283e3-b5ae-4809-8bee-8a0320e57de4-kube-api-access-7bwsk\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:36.064142 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.064104    2561 generic.go:356] "Generic (PLEG): container finished" podID="cb4283e3-b5ae-4809-8bee-8a0320e57de4" containerID="df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa" exitCode=0
Sep 08 20:27:36.064142 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.064146    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr"
Sep 08 20:27:36.064391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.064149    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr" event={"ID":"cb4283e3-b5ae-4809-8bee-8a0320e57de4","Type":"ContainerDied","Data":"df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa"}
Sep 08 20:27:36.064391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.064200    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6674f4b8-n8ntr" event={"ID":"cb4283e3-b5ae-4809-8bee-8a0320e57de4","Type":"ContainerDied","Data":"048169f0c4402895f8bd5b524b70a1e9455c61883970c4d2cf6c43d9a875851e"}
Sep 08 20:27:36.064391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.064227    2561 scope.go:117] "RemoveContainer" containerID="df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa"
Sep 08 20:27:36.072935 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.072917    2561 scope.go:117] "RemoveContainer" containerID="df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa"
Sep 08 20:27:36.073219 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:27:36.073198    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa\": container with ID starting with df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa not found: ID does not exist" containerID="df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa"
Sep 08 20:27:36.073270 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.073229    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa"} err="failed to get container status \"df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa\": rpc error: code = NotFound desc = could not find container \"df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa\": container with ID starting with df96f9eb2578fe76b7b94f277cf539eefb02aaa9b941af2bdc558a06216f4baa not found: ID does not exist"
Sep 08 20:27:36.085408 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.085383    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c6674f4b8-n8ntr"]
Sep 08 20:27:36.089218 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.089192    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c6674f4b8-n8ntr"]
Sep 08 20:27:36.362818 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:36.362723    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb4283e3-b5ae-4809-8bee-8a0320e57de4" path="/var/lib/kubelet/pods/cb4283e3-b5ae-4809-8bee-8a0320e57de4/volumes"
Sep 08 20:27:37.478032 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.477987    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c65d89f69-v8qrl" podUID="dcc7a745-bf03-4fe7-9749-8cb34707ce24" containerName="console" containerID="cri-o://9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3" gracePeriod=15
Sep 08 20:27:37.759491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.759413    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:27:37.865340 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.865309    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-serving-cert\") pod \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") "
Sep 08 20:27:37.865532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.865352    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-oauth-serving-cert\") pod \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") "
Sep 08 20:27:37.865532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.865379    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-oauth-config\") pod \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") "
Sep 08 20:27:37.865532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.865409    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-config\") pod \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") "
Sep 08 20:27:37.865532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.865444    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-service-ca\") pod \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") "
Sep 08 20:27:37.865532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.865498    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnjpl\" (UniqueName: \"kubernetes.io/projected/dcc7a745-bf03-4fe7-9749-8cb34707ce24-kube-api-access-vnjpl\") pod \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\" (UID: \"dcc7a745-bf03-4fe7-9749-8cb34707ce24\") "
Sep 08 20:27:37.865873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.865844    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-config" (OuterVolumeSpecName: "console-config") pod "dcc7a745-bf03-4fe7-9749-8cb34707ce24" (UID: "dcc7a745-bf03-4fe7-9749-8cb34707ce24"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:37.865873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.865855    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dcc7a745-bf03-4fe7-9749-8cb34707ce24" (UID: "dcc7a745-bf03-4fe7-9749-8cb34707ce24"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:37.866096 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.866077    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-service-ca" (OuterVolumeSpecName: "service-ca") pod "dcc7a745-bf03-4fe7-9749-8cb34707ce24" (UID: "dcc7a745-bf03-4fe7-9749-8cb34707ce24"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:37.867694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.867666    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dcc7a745-bf03-4fe7-9749-8cb34707ce24" (UID: "dcc7a745-bf03-4fe7-9749-8cb34707ce24"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:27:37.867804 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.867763    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dcc7a745-bf03-4fe7-9749-8cb34707ce24" (UID: "dcc7a745-bf03-4fe7-9749-8cb34707ce24"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:27:37.867804 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.867765    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcc7a745-bf03-4fe7-9749-8cb34707ce24-kube-api-access-vnjpl" (OuterVolumeSpecName: "kube-api-access-vnjpl") pod "dcc7a745-bf03-4fe7-9749-8cb34707ce24" (UID: "dcc7a745-bf03-4fe7-9749-8cb34707ce24"). InnerVolumeSpecName "kube-api-access-vnjpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:27:37.966933 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.966884    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-serving-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:37.966933 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.966928    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-oauth-serving-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:37.966933 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.966937    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-oauth-config\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:37.967158 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.966945    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-console-config\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:37.967158 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.966954    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dcc7a745-bf03-4fe7-9749-8cb34707ce24-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:37.967158 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:37.966962    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnjpl\" (UniqueName: \"kubernetes.io/projected/dcc7a745-bf03-4fe7-9749-8cb34707ce24-kube-api-access-vnjpl\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:27:38.078379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.078287    2561 generic.go:356] "Generic (PLEG): container finished" podID="dcc7a745-bf03-4fe7-9749-8cb34707ce24" containerID="9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3" exitCode=0
Sep 08 20:27:38.078379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.078341    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c65d89f69-v8qrl" event={"ID":"dcc7a745-bf03-4fe7-9749-8cb34707ce24","Type":"ContainerDied","Data":"9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3"}
Sep 08 20:27:38.078379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.078347    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c65d89f69-v8qrl"
Sep 08 20:27:38.078642 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.078419    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c65d89f69-v8qrl" event={"ID":"dcc7a745-bf03-4fe7-9749-8cb34707ce24","Type":"ContainerDied","Data":"485d41d05a3712bbd36faea68d4e1c26d0aca41cbeeda913526f6c8d88d5609a"}
Sep 08 20:27:38.078642 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.078445    2561 scope.go:117] "RemoveContainer" containerID="9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3"
Sep 08 20:27:38.087223 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.087203    2561 scope.go:117] "RemoveContainer" containerID="9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3"
Sep 08 20:27:38.087634 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:27:38.087600    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3\": container with ID starting with 9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3 not found: ID does not exist" containerID="9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3"
Sep 08 20:27:38.087680 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.087642    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3"} err="failed to get container status \"9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3\": rpc error: code = NotFound desc = could not find container \"9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3\": container with ID starting with 9beee285e7f85ee41d35e2b9739c288820a6cb8ac9910c1d2323d2c1fdc6d4a3 not found: ID does not exist"
Sep 08 20:27:38.097815 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.097790    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c65d89f69-v8qrl"]
Sep 08 20:27:38.101880 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.101855    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c65d89f69-v8qrl"]
Sep 08 20:27:38.362036 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:38.361954    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcc7a745-bf03-4fe7-9749-8cb34707ce24" path="/var/lib/kubelet/pods/dcc7a745-bf03-4fe7-9749-8cb34707ce24/volumes"
Sep 08 20:27:47.130614 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:47.130531    2561 generic.go:356] "Generic (PLEG): container finished" podID="a9733443-6da4-4bd0-99c1-0115e2aa4623" containerID="8e75daa2ae4acd0e378b016be29c9fcb74ae696162fee3982e6be627257d7b0f" exitCode=0
Sep 08 20:27:47.131003 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:47.130609    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8" event={"ID":"a9733443-6da4-4bd0-99c1-0115e2aa4623","Type":"ContainerDied","Data":"8e75daa2ae4acd0e378b016be29c9fcb74ae696162fee3982e6be627257d7b0f"}
Sep 08 20:27:47.131003 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:47.130990    2561 scope.go:117] "RemoveContainer" containerID="8e75daa2ae4acd0e378b016be29c9fcb74ae696162fee3982e6be627257d7b0f"
Sep 08 20:27:48.136574 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:27:48.136534    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-fl5q8" event={"ID":"a9733443-6da4-4bd0-99c1-0115e2aa4623","Type":"ContainerStarted","Data":"e63cfba1c203b572176f0080dc783553d3e25cc243539640fc7528dc03576f4b"}
Sep 08 20:28:34.414378 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.414335    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"]
Sep 08 20:28:34.415064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.414797    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcc7a745-bf03-4fe7-9749-8cb34707ce24" containerName="console"
Sep 08 20:28:34.415064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.414816    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcc7a745-bf03-4fe7-9749-8cb34707ce24" containerName="console"
Sep 08 20:28:34.415064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.414845    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb4283e3-b5ae-4809-8bee-8a0320e57de4" containerName="registry"
Sep 08 20:28:34.415064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.414855    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb4283e3-b5ae-4809-8bee-8a0320e57de4" containerName="registry"
Sep 08 20:28:34.415064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.414959    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb4283e3-b5ae-4809-8bee-8a0320e57de4" containerName="registry"
Sep 08 20:28:34.415064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.414972    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcc7a745-bf03-4fe7-9749-8cb34707ce24" containerName="console"
Sep 08 20:28:34.420185 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.420154    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.423396 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.423365    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 08 20:28:34.423601 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.423496    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ct652\""
Sep 08 20:28:34.424251 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.424216    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 08 20:28:34.424404 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.424230    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 08 20:28:34.424548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.424527    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 08 20:28:34.424668 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.424539    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 08 20:28:34.430078 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.430051    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 08 20:28:34.434862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.434831    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"]
Sep 08 20:28:34.522875 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.522840    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-federate-client-tls\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.522875 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.522881    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-metrics-client-ca\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.523111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.523052    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-serving-certs-ca-bundle\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.523154 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.523125    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-secret-telemeter-client\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.523212 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.523162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.523212 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.523186    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.523317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.523215    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-telemeter-client-tls\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.523317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.523242    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4klws\" (UniqueName: \"kubernetes.io/projected/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-kube-api-access-4klws\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.623768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.623726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-serving-certs-ca-bundle\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.623980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.623782    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-secret-telemeter-client\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.623980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.623807    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.623980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.623823    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.623980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.623843    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-telemeter-client-tls\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.623980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.623862    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4klws\" (UniqueName: \"kubernetes.io/projected/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-kube-api-access-4klws\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.623980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.623881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-federate-client-tls\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.623980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.623932    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-metrics-client-ca\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.624652 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.624621    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-serving-certs-ca-bundle\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.624796 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.624709    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.625038 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.625017    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-metrics-client-ca\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.626627 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.626600    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.626715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.626627    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-secret-telemeter-client\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.626998 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.626980    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-telemeter-client-tls\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.627043 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.626980    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-federate-client-tls\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.632679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.632660    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4klws\" (UniqueName: \"kubernetes.io/projected/476785d2-a0aa-4fe9-8ac1-e0070e9d12bd-kube-api-access-4klws\") pod \"telemeter-client-69cbfcd64f-fbdcr\" (UID: \"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd\") " pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:34.732914 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:34.732811    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"
Sep 08 20:28:35.011256 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:35.011225    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr"]
Sep 08 20:28:35.013676 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:28:35.013648    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476785d2_a0aa_4fe9_8ac1_e0070e9d12bd.slice/crio-c1140aad3c70f7a24f74237cba2578d28c38f019eab94d55d307a782b0408cb9 WatchSource:0}: Error finding container c1140aad3c70f7a24f74237cba2578d28c38f019eab94d55d307a782b0408cb9: Status 404 returned error can't find the container with id c1140aad3c70f7a24f74237cba2578d28c38f019eab94d55d307a782b0408cb9
Sep 08 20:28:35.314368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:35.314277    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr" event={"ID":"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd","Type":"ContainerStarted","Data":"c1140aad3c70f7a24f74237cba2578d28c38f019eab94d55d307a782b0408cb9"}
Sep 08 20:28:37.325740 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:37.325705    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr" event={"ID":"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd","Type":"ContainerStarted","Data":"63966d54ea2046c92ca5450307da567bdd6316eedfc9ce55db69c1a09ac53018"}
Sep 08 20:28:37.326107 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:37.325764    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr" event={"ID":"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd","Type":"ContainerStarted","Data":"5b48c1dc3a42092a640e50ae0fa8f397550a16af10356968e1b7e2bf2573e41c"}
Sep 08 20:28:38.331204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.331162    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr" event={"ID":"476785d2-a0aa-4fe9-8ac1-e0070e9d12bd","Type":"ContainerStarted","Data":"e1fd94d872a80c94035b0bd4e97871fe1e4e6dc87cbb8cf6a08d50d72ccf7ebb"}
Sep 08 20:28:38.354768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.354710    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69cbfcd64f-fbdcr" podStartSLOduration=2.212220427 podStartE2EDuration="4.354690395s" totalImagesPullingTime="2.142469968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:28:38.352629955 +0000 UTC m=+204.517800531" watchObservedRunningTime="2026-09-08 20:28:38.354690395 +0000 UTC m=+204.519860973"
Sep 08 20:28:38.434993 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.434949    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c774d648-bljwd"]
Sep 08 20:28:38.438772 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.438744    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.443264 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.442674    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 20:28:38.443264 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.442975    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 20:28:38.443264 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.443179    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 20:28:38.443540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.443394    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 20:28:38.443540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.443456    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 20:28:38.443540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.443495    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zlxsw\""
Sep 08 20:28:38.447981 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.447957    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 08 20:28:38.449773 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.449682    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c774d648-bljwd"]
Sep 08 20:28:38.553996 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.553959    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8xp7\" (UniqueName: \"kubernetes.io/projected/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-kube-api-access-z8xp7\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.554185 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.554001    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-service-ca\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.554185 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.554063    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-serving-cert\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.554185 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.554101    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-oauth-config\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.554185 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.554143    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-config\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.554355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.554211    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-oauth-serving-cert\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.554355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.554232    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-trusted-ca-bundle\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.655036 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.654997    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8xp7\" (UniqueName: \"kubernetes.io/projected/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-kube-api-access-z8xp7\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.655243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.655049    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-service-ca\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.655243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.655219    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-serving-cert\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.655358 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.655292    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-oauth-config\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.655358 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.655321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-config\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.655454 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.655404    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-oauth-serving-cert\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.655454 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.655431    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-trusted-ca-bundle\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.656172 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.655832    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-service-ca\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.656322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.656297    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-trusted-ca-bundle\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.656396 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.656374    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-config\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.656989 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.656963    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-oauth-serving-cert\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.658823 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.658797    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-oauth-config\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.659012 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.658994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-serving-cert\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.663010 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.662991    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8xp7\" (UniqueName: \"kubernetes.io/projected/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-kube-api-access-z8xp7\") pod \"console-7c774d648-bljwd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") " pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:38.753453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:38.753421    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:39.036691 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:39.036601    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c774d648-bljwd"]
Sep 08 20:28:39.039461 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:28:39.039431    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d625bc1_ce3d_409f_b51d_2f842b3c8ddd.slice/crio-1990ab356b079529a610975ded90a87001c96f1f730b5c7bdeb29acae38282d2 WatchSource:0}: Error finding container 1990ab356b079529a610975ded90a87001c96f1f730b5c7bdeb29acae38282d2: Status 404 returned error can't find the container with id 1990ab356b079529a610975ded90a87001c96f1f730b5c7bdeb29acae38282d2
Sep 08 20:28:39.336981 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:39.336876    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c774d648-bljwd" event={"ID":"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd","Type":"ContainerStarted","Data":"bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f"}
Sep 08 20:28:39.336981 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:39.336944    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c774d648-bljwd" event={"ID":"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd","Type":"ContainerStarted","Data":"1990ab356b079529a610975ded90a87001c96f1f730b5c7bdeb29acae38282d2"}
Sep 08 20:28:39.355770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:39.355711    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c774d648-bljwd" podStartSLOduration=1.355692956 podStartE2EDuration="1.355692956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:28:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:28:39.354582963 +0000 UTC m=+205.519753540" watchObservedRunningTime="2026-09-08 20:28:39.355692956 +0000 UTC m=+205.520863531"
Sep 08 20:28:48.753940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:48.753875    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:48.754531 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:48.753968    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:48.758822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:48.758798    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:28:49.149660 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:49.149578    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c774d648-bljwd"]
Sep 08 20:28:49.380309 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:28:49.380278    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:29:15.401569 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.401505    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c774d648-bljwd" podUID="4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" containerName="console" containerID="cri-o://bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f" gracePeriod=15
Sep 08 20:29:15.640212 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.640187    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:29:15.765113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765014    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-oauth-serving-cert\") pod \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") "
Sep 08 20:29:15.765113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765098    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-oauth-config\") pod \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") "
Sep 08 20:29:15.765113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765119    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-config\") pod \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") "
Sep 08 20:29:15.765395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765138    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-serving-cert\") pod \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") "
Sep 08 20:29:15.765395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765182    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-service-ca\") pod \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") "
Sep 08 20:29:15.765395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765219    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-trusted-ca-bundle\") pod \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") "
Sep 08 20:29:15.765395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765248    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8xp7\" (UniqueName: \"kubernetes.io/projected/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-kube-api-access-z8xp7\") pod \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\" (UID: \"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd\") "
Sep 08 20:29:15.765608 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765549    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" (UID: "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:29:15.765661 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765593    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-config" (OuterVolumeSpecName: "console-config") pod "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" (UID: "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:29:15.765753 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765658    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-service-ca" (OuterVolumeSpecName: "service-ca") pod "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" (UID: "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:29:15.765889 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.765861    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" (UID: "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:29:15.767442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.767419    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" (UID: "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:29:15.767442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.767419    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-kube-api-access-z8xp7" (OuterVolumeSpecName: "kube-api-access-z8xp7") pod "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" (UID: "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd"). InnerVolumeSpecName "kube-api-access-z8xp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:29:15.767565 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.767436    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" (UID: "4d625bc1-ce3d-409f-b51d-2f842b3c8ddd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:29:15.865845 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.865805    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-trusted-ca-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:29:15.865845 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.865836    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8xp7\" (UniqueName: \"kubernetes.io/projected/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-kube-api-access-z8xp7\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:29:15.865845 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.865844    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-oauth-serving-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:29:15.865845 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.865853    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-oauth-config\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:29:15.866137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.865862    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-config\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:29:15.866137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.865870    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-console-serving-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:29:15.866137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:15.865878    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:29:16.484845 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.484810    2561 generic.go:356] "Generic (PLEG): container finished" podID="4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" containerID="bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f" exitCode=0
Sep 08 20:29:16.485377 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.484864    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c774d648-bljwd"
Sep 08 20:29:16.485377 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.484916    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c774d648-bljwd" event={"ID":"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd","Type":"ContainerDied","Data":"bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f"}
Sep 08 20:29:16.485377 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.484970    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c774d648-bljwd" event={"ID":"4d625bc1-ce3d-409f-b51d-2f842b3c8ddd","Type":"ContainerDied","Data":"1990ab356b079529a610975ded90a87001c96f1f730b5c7bdeb29acae38282d2"}
Sep 08 20:29:16.485377 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.484990    2561 scope.go:117] "RemoveContainer" containerID="bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f"
Sep 08 20:29:16.493435 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.493413    2561 scope.go:117] "RemoveContainer" containerID="bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f"
Sep 08 20:29:16.493788 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:29:16.493769    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f\": container with ID starting with bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f not found: ID does not exist" containerID="bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f"
Sep 08 20:29:16.493829 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.493798    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f"} err="failed to get container status \"bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f\": rpc error: code = NotFound desc = could not find container \"bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f\": container with ID starting with bea1670e42122e4a251db1c2078e4c96daece3d6e4458fdd35d5c6c83d44636f not found: ID does not exist"
Sep 08 20:29:16.501192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.501163    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c774d648-bljwd"]
Sep 08 20:29:16.505486 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:16.505453    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c774d648-bljwd"]
Sep 08 20:29:18.361996 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:18.361961    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" path="/var/lib/kubelet/pods/4d625bc1-ce3d-409f-b51d-2f842b3c8ddd/volumes"
Sep 08 20:29:57.907977 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.907927    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"]
Sep 08 20:29:57.908554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.908468    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" containerName="console"
Sep 08 20:29:57.908554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.908490    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" containerName="console"
Sep 08 20:29:57.908680 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.908581    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d625bc1-ce3d-409f-b51d-2f842b3c8ddd" containerName="console"
Sep 08 20:29:57.911741 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.911717    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:57.917604 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.917572    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 20:29:57.918348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.918310    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-49dhb\""
Sep 08 20:29:57.918645 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.918622    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 20:29:57.920738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:57.920710    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"]
Sep 08 20:29:58.004862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.004819    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhbzf\" (UniqueName: \"kubernetes.io/projected/4d43ae36-e30e-48bf-8f64-38ced0c5b009-kube-api-access-zhbzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.005064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.004876    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.005064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.004918    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.105696 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.105656    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhbzf\" (UniqueName: \"kubernetes.io/projected/4d43ae36-e30e-48bf-8f64-38ced0c5b009-kube-api-access-zhbzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.105830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.105710    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.105830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.105728    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.106125 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.106106    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.106197 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.106156    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.114435 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.114413    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhbzf\" (UniqueName: \"kubernetes.io/projected/4d43ae36-e30e-48bf-8f64-38ced0c5b009-kube-api-access-zhbzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.227740 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.227654    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:29:58.497181 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.497059    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"]
Sep 08 20:29:58.499421 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:29:58.499387    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d43ae36_e30e_48bf_8f64_38ced0c5b009.slice/crio-3ab5c422f438df35883397fe5e52d82b91947a96f0653171efc1e3ce16a6f449 WatchSource:0}: Error finding container 3ab5c422f438df35883397fe5e52d82b91947a96f0653171efc1e3ce16a6f449: Status 404 returned error can't find the container with id 3ab5c422f438df35883397fe5e52d82b91947a96f0653171efc1e3ce16a6f449
Sep 08 20:29:58.646297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:29:58.646264    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6" event={"ID":"4d43ae36-e30e-48bf-8f64-38ced0c5b009","Type":"ContainerStarted","Data":"3ab5c422f438df35883397fe5e52d82b91947a96f0653171efc1e3ce16a6f449"}
Sep 08 20:30:04.676260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:04.676163    2561 generic.go:356] "Generic (PLEG): container finished" podID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerID="a194cd78abf917fa08b9295737d056da3576b02e4ef06c602dfe0908ec0b404d" exitCode=0
Sep 08 20:30:04.676701 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:04.676253    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6" event={"ID":"4d43ae36-e30e-48bf-8f64-38ced0c5b009","Type":"ContainerDied","Data":"a194cd78abf917fa08b9295737d056da3576b02e4ef06c602dfe0908ec0b404d"}
Sep 08 20:30:07.691447 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:07.691413    2561 generic.go:356] "Generic (PLEG): container finished" podID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerID="23a69d34f55df9dae7dafbceece847efd8dd625cf873cff00032bab66424c3cd" exitCode=0
Sep 08 20:30:07.691864 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:07.691498    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6" event={"ID":"4d43ae36-e30e-48bf-8f64-38ced0c5b009","Type":"ContainerDied","Data":"23a69d34f55df9dae7dafbceece847efd8dd625cf873cff00032bab66424c3cd"}
Sep 08 20:30:14.270790 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:14.270753    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:30:14.271249 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:14.270917    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:30:14.282180 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:14.282157    2561 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 20:30:14.722721 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:14.722683    2561 generic.go:356] "Generic (PLEG): container finished" podID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerID="660d9e3e1f8659aad2eed20d31abeaaa9eeaa00a8a72bb68cfe5a43f52353489" exitCode=0
Sep 08 20:30:14.722943 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:14.722769    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6" event={"ID":"4d43ae36-e30e-48bf-8f64-38ced0c5b009","Type":"ContainerDied","Data":"660d9e3e1f8659aad2eed20d31abeaaa9eeaa00a8a72bb68cfe5a43f52353489"}
Sep 08 20:30:15.851491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:15.851466    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:30:15.964241 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:15.964208    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-util\") pod \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") "
Sep 08 20:30:15.964241 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:15.964248    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhbzf\" (UniqueName: \"kubernetes.io/projected/4d43ae36-e30e-48bf-8f64-38ced0c5b009-kube-api-access-zhbzf\") pod \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") "
Sep 08 20:30:15.964477 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:15.964323    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-bundle\") pod \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\" (UID: \"4d43ae36-e30e-48bf-8f64-38ced0c5b009\") "
Sep 08 20:30:15.965038 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:15.965005    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-bundle" (OuterVolumeSpecName: "bundle") pod "4d43ae36-e30e-48bf-8f64-38ced0c5b009" (UID: "4d43ae36-e30e-48bf-8f64-38ced0c5b009"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:30:15.966609 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:15.966586    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d43ae36-e30e-48bf-8f64-38ced0c5b009-kube-api-access-zhbzf" (OuterVolumeSpecName: "kube-api-access-zhbzf") pod "4d43ae36-e30e-48bf-8f64-38ced0c5b009" (UID: "4d43ae36-e30e-48bf-8f64-38ced0c5b009"). InnerVolumeSpecName "kube-api-access-zhbzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:30:15.969064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:15.969040    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-util" (OuterVolumeSpecName: "util") pod "4d43ae36-e30e-48bf-8f64-38ced0c5b009" (UID: "4d43ae36-e30e-48bf-8f64-38ced0c5b009"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:30:16.065224 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:16.065128    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:16.065224 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:16.065173    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d43ae36-e30e-48bf-8f64-38ced0c5b009-util\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:16.065224 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:16.065184    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhbzf\" (UniqueName: \"kubernetes.io/projected/4d43ae36-e30e-48bf-8f64-38ced0c5b009-kube-api-access-zhbzf\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:16.732169 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:16.732132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6" event={"ID":"4d43ae36-e30e-48bf-8f64-38ced0c5b009","Type":"ContainerDied","Data":"3ab5c422f438df35883397fe5e52d82b91947a96f0653171efc1e3ce16a6f449"}
Sep 08 20:30:16.732169 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:16.732159    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jf2c6"
Sep 08 20:30:16.732413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:16.732180    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ab5c422f438df35883397fe5e52d82b91947a96f0653171efc1e3ce16a6f449"
Sep 08 20:30:20.882880 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.882837    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"]
Sep 08 20:30:20.884056 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.884030    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerName="extract"
Sep 08 20:30:20.884190 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.884179    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerName="extract"
Sep 08 20:30:20.884291 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.884281    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerName="pull"
Sep 08 20:30:20.884357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.884349    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerName="pull"
Sep 08 20:30:20.884452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.884444    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerName="util"
Sep 08 20:30:20.884516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.884509    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerName="util"
Sep 08 20:30:20.885089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.884937    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d43ae36-e30e-48bf-8f64-38ced0c5b009" containerName="extract"
Sep 08 20:30:20.917583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.917546    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"]
Sep 08 20:30:20.917749 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.917671    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"
Sep 08 20:30:20.922209 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.922172    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:30:20.922360 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.922291    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 08 20:30:20.934332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:20.934311    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ppp8c\""
Sep 08 20:30:21.006066 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.006029    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/735fbecf-1463-4992-a895-f9d49100b5c7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ms5z4\" (UID: \"735fbecf-1463-4992-a895-f9d49100b5c7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"
Sep 08 20:30:21.006066 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.006068    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwsjg\" (UniqueName: \"kubernetes.io/projected/735fbecf-1463-4992-a895-f9d49100b5c7-kube-api-access-lwsjg\") pod \"cert-manager-operator-controller-manager-55c444bf86-ms5z4\" (UID: \"735fbecf-1463-4992-a895-f9d49100b5c7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"
Sep 08 20:30:21.106838 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.106793    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/735fbecf-1463-4992-a895-f9d49100b5c7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ms5z4\" (UID: \"735fbecf-1463-4992-a895-f9d49100b5c7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"
Sep 08 20:30:21.107048 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.106848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwsjg\" (UniqueName: \"kubernetes.io/projected/735fbecf-1463-4992-a895-f9d49100b5c7-kube-api-access-lwsjg\") pod \"cert-manager-operator-controller-manager-55c444bf86-ms5z4\" (UID: \"735fbecf-1463-4992-a895-f9d49100b5c7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"
Sep 08 20:30:21.107201 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.107178    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/735fbecf-1463-4992-a895-f9d49100b5c7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ms5z4\" (UID: \"735fbecf-1463-4992-a895-f9d49100b5c7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"
Sep 08 20:30:21.116053 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.116019    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwsjg\" (UniqueName: \"kubernetes.io/projected/735fbecf-1463-4992-a895-f9d49100b5c7-kube-api-access-lwsjg\") pod \"cert-manager-operator-controller-manager-55c444bf86-ms5z4\" (UID: \"735fbecf-1463-4992-a895-f9d49100b5c7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"
Sep 08 20:30:21.226445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.226405    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"
Sep 08 20:30:21.522591 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.522483    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4"]
Sep 08 20:30:21.525877 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:30:21.525847    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735fbecf_1463_4992_a895_f9d49100b5c7.slice/crio-a82df9905e98c4a4be66deb7e036dd68c562a8a9196a88ee04c3bb3acdcb7248 WatchSource:0}: Error finding container a82df9905e98c4a4be66deb7e036dd68c562a8a9196a88ee04c3bb3acdcb7248: Status 404 returned error can't find the container with id a82df9905e98c4a4be66deb7e036dd68c562a8a9196a88ee04c3bb3acdcb7248
Sep 08 20:30:21.528825 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.528804    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:30:21.753175 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:21.753135    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4" event={"ID":"735fbecf-1463-4992-a895-f9d49100b5c7","Type":"ContainerStarted","Data":"a82df9905e98c4a4be66deb7e036dd68c562a8a9196a88ee04c3bb3acdcb7248"}
Sep 08 20:30:24.768340 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:24.768300    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4" event={"ID":"735fbecf-1463-4992-a895-f9d49100b5c7","Type":"ContainerStarted","Data":"ce34b5055e4c769ae837091b1cf81476109d26491eab10521709d35120876d10"}
Sep 08 20:30:24.792220 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:24.792167    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ms5z4" podStartSLOduration=2.278543319 podStartE2EDuration="4.792148856s" totalImagesPullingTime="2.513605537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:30:24.790035233 +0000 UTC m=+310.955205810" watchObservedRunningTime="2026-09-08 20:30:24.792148856 +0000 UTC m=+310.957319431"
Sep 08 20:30:25.151181 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.151156    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-54wp9"]
Sep 08 20:30:25.177672 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.177642    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-54wp9"]
Sep 08 20:30:25.177816 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.177738    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:25.181352 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.181325    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sh2td\""
Sep 08 20:30:25.181532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.181362    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 08 20:30:25.181532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.181417    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 08 20:30:25.345833 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.345795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eeb4d1b8-1ea9-4103-a5bc-e398183f08bc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-54wp9\" (UID: \"eeb4d1b8-1ea9-4103-a5bc-e398183f08bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:25.346080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.345870    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnfp7\" (UniqueName: \"kubernetes.io/projected/eeb4d1b8-1ea9-4103-a5bc-e398183f08bc-kube-api-access-nnfp7\") pod \"cert-manager-webhook-6486ffd997-54wp9\" (UID: \"eeb4d1b8-1ea9-4103-a5bc-e398183f08bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:25.446432 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.446333    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnfp7\" (UniqueName: \"kubernetes.io/projected/eeb4d1b8-1ea9-4103-a5bc-e398183f08bc-kube-api-access-nnfp7\") pod \"cert-manager-webhook-6486ffd997-54wp9\" (UID: \"eeb4d1b8-1ea9-4103-a5bc-e398183f08bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:25.446432 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.446426    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eeb4d1b8-1ea9-4103-a5bc-e398183f08bc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-54wp9\" (UID: \"eeb4d1b8-1ea9-4103-a5bc-e398183f08bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:25.469033 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.469000    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eeb4d1b8-1ea9-4103-a5bc-e398183f08bc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-54wp9\" (UID: \"eeb4d1b8-1ea9-4103-a5bc-e398183f08bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:25.469825 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.469801    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnfp7\" (UniqueName: \"kubernetes.io/projected/eeb4d1b8-1ea9-4103-a5bc-e398183f08bc-kube-api-access-nnfp7\") pod \"cert-manager-webhook-6486ffd997-54wp9\" (UID: \"eeb4d1b8-1ea9-4103-a5bc-e398183f08bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:25.496972 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.496939    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:25.507112 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.507079    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"]
Sep 08 20:30:25.523169 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.523133    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.526683 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.526656    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"]
Sep 08 20:30:25.527201 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.527176    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-49dhb\""
Sep 08 20:30:25.527338 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.527192    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 20:30:25.527501 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.527479    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 20:30:25.647789 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.647745    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.647789 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.647795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.648031 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.647919    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jtj5\" (UniqueName: \"kubernetes.io/projected/79a81589-5378-4c71-9e6f-e3d0918aa5f2-kube-api-access-2jtj5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.749296 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.749216    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.749296 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.749259    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.749482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.749325    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jtj5\" (UniqueName: \"kubernetes.io/projected/79a81589-5378-4c71-9e6f-e3d0918aa5f2-kube-api-access-2jtj5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.749686 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.749662    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.749720 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.749671    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.759880 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.759845    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jtj5\" (UniqueName: \"kubernetes.io/projected/79a81589-5378-4c71-9e6f-e3d0918aa5f2-kube-api-access-2jtj5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:25.791006 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.790909    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-54wp9"]
Sep 08 20:30:25.793698 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:30:25.793664    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeeb4d1b8_1ea9_4103_a5bc_e398183f08bc.slice/crio-fb3ee5e85757518d466838de8a457fe55715e08a7e079e2b3cc0a535693203d2 WatchSource:0}: Error finding container fb3ee5e85757518d466838de8a457fe55715e08a7e079e2b3cc0a535693203d2: Status 404 returned error can't find the container with id fb3ee5e85757518d466838de8a457fe55715e08a7e079e2b3cc0a535693203d2
Sep 08 20:30:25.844644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:25.844608    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:26.141734 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:26.141709    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"]
Sep 08 20:30:26.143994 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:30:26.143965    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a81589_5378_4c71_9e6f_e3d0918aa5f2.slice/crio-63d1f654497cbd096a36a048bcb16fdace9d9b37a33f474eb792c11c984dc54f WatchSource:0}: Error finding container 63d1f654497cbd096a36a048bcb16fdace9d9b37a33f474eb792c11c984dc54f: Status 404 returned error can't find the container with id 63d1f654497cbd096a36a048bcb16fdace9d9b37a33f474eb792c11c984dc54f
Sep 08 20:30:26.779861 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:26.779818    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9" event={"ID":"eeb4d1b8-1ea9-4103-a5bc-e398183f08bc","Type":"ContainerStarted","Data":"fb3ee5e85757518d466838de8a457fe55715e08a7e079e2b3cc0a535693203d2"}
Sep 08 20:30:26.781714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:26.781681    2561 generic.go:356] "Generic (PLEG): container finished" podID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerID="27794317045a5a7b271fb6c4109150f16b0338072b6fc2a341e17d8a4be20bc7" exitCode=0
Sep 08 20:30:26.781852 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:26.781773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4" event={"ID":"79a81589-5378-4c71-9e6f-e3d0918aa5f2","Type":"ContainerDied","Data":"27794317045a5a7b271fb6c4109150f16b0338072b6fc2a341e17d8a4be20bc7"}
Sep 08 20:30:26.781852 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:26.781824    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4" event={"ID":"79a81589-5378-4c71-9e6f-e3d0918aa5f2","Type":"ContainerStarted","Data":"63d1f654497cbd096a36a048bcb16fdace9d9b37a33f474eb792c11c984dc54f"}
Sep 08 20:30:29.797744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:29.797648    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9" event={"ID":"eeb4d1b8-1ea9-4103-a5bc-e398183f08bc","Type":"ContainerStarted","Data":"3afae3736ab395b2cbd886e1dbcfa261a1836368fdf8641e471ef95e6f7b51b1"}
Sep 08 20:30:29.798202 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:29.797772    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:29.799595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:29.799572    2561 generic.go:356] "Generic (PLEG): container finished" podID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerID="9f5bb3d413befee118351421d368a7f3b7c98e9a4536235c8a56bea9ffbd1f5d" exitCode=0
Sep 08 20:30:29.799692 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:29.799633    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4" event={"ID":"79a81589-5378-4c71-9e6f-e3d0918aa5f2","Type":"ContainerDied","Data":"9f5bb3d413befee118351421d368a7f3b7c98e9a4536235c8a56bea9ffbd1f5d"}
Sep 08 20:30:29.814802 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:29.814741    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9" podStartSLOduration=1.16525595 podStartE2EDuration="4.814719776s" totalImagesPullingTime="3.649463826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:30:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:30:29.813234989 +0000 UTC m=+315.978405566" watchObservedRunningTime="2026-09-08 20:30:29.814719776 +0000 UTC m=+315.979890354"
Sep 08 20:30:30.805635 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:30.805601    2561 generic.go:356] "Generic (PLEG): container finished" podID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerID="8750ab2dad4312f7cb370d843a28c45afd094dc42e0d75d0f89182febf9ca195" exitCode=0
Sep 08 20:30:30.806050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:30.805683    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4" event={"ID":"79a81589-5378-4c71-9e6f-e3d0918aa5f2","Type":"ContainerDied","Data":"8750ab2dad4312f7cb370d843a28c45afd094dc42e0d75d0f89182febf9ca195"}
Sep 08 20:30:31.936064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:31.936040    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:32.104942 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.104820    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-bundle\") pod \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") "
Sep 08 20:30:32.104942 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.104916    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-util\") pod \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") "
Sep 08 20:30:32.105131 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.104959    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jtj5\" (UniqueName: \"kubernetes.io/projected/79a81589-5378-4c71-9e6f-e3d0918aa5f2-kube-api-access-2jtj5\") pod \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\" (UID: \"79a81589-5378-4c71-9e6f-e3d0918aa5f2\") "
Sep 08 20:30:32.105260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.105236    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-bundle" (OuterVolumeSpecName: "bundle") pod "79a81589-5378-4c71-9e6f-e3d0918aa5f2" (UID: "79a81589-5378-4c71-9e6f-e3d0918aa5f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:30:32.107070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.107035    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a81589-5378-4c71-9e6f-e3d0918aa5f2-kube-api-access-2jtj5" (OuterVolumeSpecName: "kube-api-access-2jtj5") pod "79a81589-5378-4c71-9e6f-e3d0918aa5f2" (UID: "79a81589-5378-4c71-9e6f-e3d0918aa5f2"). InnerVolumeSpecName "kube-api-access-2jtj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:30:32.109823 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.109802    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-util" (OuterVolumeSpecName: "util") pod "79a81589-5378-4c71-9e6f-e3d0918aa5f2" (UID: "79a81589-5378-4c71-9e6f-e3d0918aa5f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:30:32.206110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.206054    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:32.206110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.206100    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79a81589-5378-4c71-9e6f-e3d0918aa5f2-util\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:32.206110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.206111    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jtj5\" (UniqueName: \"kubernetes.io/projected/79a81589-5378-4c71-9e6f-e3d0918aa5f2-kube-api-access-2jtj5\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:32.769445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.769400    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"]
Sep 08 20:30:32.769741 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.769727    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerName="util"
Sep 08 20:30:32.769784 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.769743    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerName="util"
Sep 08 20:30:32.769784 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.769758    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerName="pull"
Sep 08 20:30:32.769784 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.769763    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerName="pull"
Sep 08 20:30:32.769784 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.769782    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerName="extract"
Sep 08 20:30:32.769945 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.769787    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerName="extract"
Sep 08 20:30:32.769945 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.769835    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="79a81589-5378-4c71-9e6f-e3d0918aa5f2" containerName="extract"
Sep 08 20:30:32.773785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.773763    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"
Sep 08 20:30:32.776220 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.776203    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-rcwkq\""
Sep 08 20:30:32.780555 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.780428    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"]
Sep 08 20:30:32.815922 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.815882    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4"
Sep 08 20:30:32.815922 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.815885    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv66t4" event={"ID":"79a81589-5378-4c71-9e6f-e3d0918aa5f2","Type":"ContainerDied","Data":"63d1f654497cbd096a36a048bcb16fdace9d9b37a33f474eb792c11c984dc54f"}
Sep 08 20:30:32.816131 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.815948    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63d1f654497cbd096a36a048bcb16fdace9d9b37a33f474eb792c11c984dc54f"
Sep 08 20:30:32.912048 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.912003    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8vrc\" (UniqueName: \"kubernetes.io/projected/55730f19-b005-43a5-9ca5-ec9d1b16a930-kube-api-access-h8vrc\") pod \"cert-manager-cainjector-559cdd8c87-tpv9k\" (UID: \"55730f19-b005-43a5-9ca5-ec9d1b16a930\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"
Sep 08 20:30:32.912048 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:32.912056    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55730f19-b005-43a5-9ca5-ec9d1b16a930-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tpv9k\" (UID: \"55730f19-b005-43a5-9ca5-ec9d1b16a930\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"
Sep 08 20:30:33.012676 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.012640    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vrc\" (UniqueName: \"kubernetes.io/projected/55730f19-b005-43a5-9ca5-ec9d1b16a930-kube-api-access-h8vrc\") pod \"cert-manager-cainjector-559cdd8c87-tpv9k\" (UID: \"55730f19-b005-43a5-9ca5-ec9d1b16a930\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"
Sep 08 20:30:33.013135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.012688    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55730f19-b005-43a5-9ca5-ec9d1b16a930-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tpv9k\" (UID: \"55730f19-b005-43a5-9ca5-ec9d1b16a930\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"
Sep 08 20:30:33.021675 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.021606    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55730f19-b005-43a5-9ca5-ec9d1b16a930-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tpv9k\" (UID: \"55730f19-b005-43a5-9ca5-ec9d1b16a930\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"
Sep 08 20:30:33.021791 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.021769    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8vrc\" (UniqueName: \"kubernetes.io/projected/55730f19-b005-43a5-9ca5-ec9d1b16a930-kube-api-access-h8vrc\") pod \"cert-manager-cainjector-559cdd8c87-tpv9k\" (UID: \"55730f19-b005-43a5-9ca5-ec9d1b16a930\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"
Sep 08 20:30:33.083667 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.083628    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"
Sep 08 20:30:33.357136 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.357060    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k"]
Sep 08 20:30:33.360351 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:30:33.360320    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55730f19_b005_43a5_9ca5_ec9d1b16a930.slice/crio-3ea67090e5e1a06943adc432df856d02c5477e033c1d9ea6556d0f13ddb642f8 WatchSource:0}: Error finding container 3ea67090e5e1a06943adc432df856d02c5477e033c1d9ea6556d0f13ddb642f8: Status 404 returned error can't find the container with id 3ea67090e5e1a06943adc432df856d02c5477e033c1d9ea6556d0f13ddb642f8
Sep 08 20:30:33.821965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.821928    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k" event={"ID":"55730f19-b005-43a5-9ca5-ec9d1b16a930","Type":"ContainerStarted","Data":"fb2c0d4c25685cba2573a82d17617ab355d896ca9dfe7955a2151104ef4d59d9"}
Sep 08 20:30:33.821965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.821970    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k" event={"ID":"55730f19-b005-43a5-9ca5-ec9d1b16a930","Type":"ContainerStarted","Data":"3ea67090e5e1a06943adc432df856d02c5477e033c1d9ea6556d0f13ddb642f8"}
Sep 08 20:30:33.842330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:33.842280    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tpv9k" podStartSLOduration=1.842264009 podStartE2EDuration="1.842264009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:30:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:30:33.840989207 +0000 UTC m=+320.006159783" watchObservedRunningTime="2026-09-08 20:30:33.842264009 +0000 UTC m=+320.007434585"
Sep 08 20:30:35.808774 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:35.808743    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-54wp9"
Sep 08 20:30:44.542052 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.542020    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"]
Sep 08 20:30:44.545623 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.545601    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.548095 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.548074    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-49dhb\""
Sep 08 20:30:44.548214 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.548112    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 20:30:44.549158 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.549137    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 20:30:44.552308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.552289    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"]
Sep 08 20:30:44.603355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.603316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.603521 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.603362    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.603521 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.603416    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvtrx\" (UniqueName: \"kubernetes.io/projected/4c986b9f-85b6-4e37-8eec-19d81894140d-kube-api-access-hvtrx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.703880 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.703839    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.704065 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.703887    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.704065 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.703944    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvtrx\" (UniqueName: \"kubernetes.io/projected/4c986b9f-85b6-4e37-8eec-19d81894140d-kube-api-access-hvtrx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.704376 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.704351    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.704413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.704361    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.711992 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.711965    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvtrx\" (UniqueName: \"kubernetes.io/projected/4c986b9f-85b6-4e37-8eec-19d81894140d-kube-api-access-hvtrx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:44.856648 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:44.856564    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:45.127753 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:45.127727    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"]
Sep 08 20:30:45.129485 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:30:45.129454    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c986b9f_85b6_4e37_8eec_19d81894140d.slice/crio-7cfcea31dc5cbda928c3ccc8a95dd015793ac6b616bff402821406fab0c3a3e4 WatchSource:0}: Error finding container 7cfcea31dc5cbda928c3ccc8a95dd015793ac6b616bff402821406fab0c3a3e4: Status 404 returned error can't find the container with id 7cfcea31dc5cbda928c3ccc8a95dd015793ac6b616bff402821406fab0c3a3e4
Sep 08 20:30:45.885399 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:45.885363    2561 generic.go:356] "Generic (PLEG): container finished" podID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerID="6d44d8316aaa24497e4978e0572a870340627526743c60e44ff67fcb6b99746f" exitCode=0
Sep 08 20:30:45.885764 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:45.885448    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t" event={"ID":"4c986b9f-85b6-4e37-8eec-19d81894140d","Type":"ContainerDied","Data":"6d44d8316aaa24497e4978e0572a870340627526743c60e44ff67fcb6b99746f"}
Sep 08 20:30:45.885764 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:45.885501    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t" event={"ID":"4c986b9f-85b6-4e37-8eec-19d81894140d","Type":"ContainerStarted","Data":"7cfcea31dc5cbda928c3ccc8a95dd015793ac6b616bff402821406fab0c3a3e4"}
Sep 08 20:30:46.892069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:46.891983    2561 generic.go:356] "Generic (PLEG): container finished" podID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerID="a70040fd675c4ed1e4dc1678e10337e17f3b40421f7328d71a58d9aa0b9068ca" exitCode=0
Sep 08 20:30:46.892069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:46.892054    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t" event={"ID":"4c986b9f-85b6-4e37-8eec-19d81894140d","Type":"ContainerDied","Data":"a70040fd675c4ed1e4dc1678e10337e17f3b40421f7328d71a58d9aa0b9068ca"}
Sep 08 20:30:47.898981 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:47.898945    2561 generic.go:356] "Generic (PLEG): container finished" podID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerID="f4a5969e40b0fa47761a68d54239c92146252761d5c6118409e8e66cea32d19d" exitCode=0
Sep 08 20:30:47.899386 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:47.899021    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t" event={"ID":"4c986b9f-85b6-4e37-8eec-19d81894140d","Type":"ContainerDied","Data":"f4a5969e40b0fa47761a68d54239c92146252761d5c6118409e8e66cea32d19d"}
Sep 08 20:30:49.032089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.032054    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:49.133859 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.133819    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvtrx\" (UniqueName: \"kubernetes.io/projected/4c986b9f-85b6-4e37-8eec-19d81894140d-kube-api-access-hvtrx\") pod \"4c986b9f-85b6-4e37-8eec-19d81894140d\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") "
Sep 08 20:30:49.134052 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.133874    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-util\") pod \"4c986b9f-85b6-4e37-8eec-19d81894140d\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") "
Sep 08 20:30:49.134052 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.133912    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-bundle\") pod \"4c986b9f-85b6-4e37-8eec-19d81894140d\" (UID: \"4c986b9f-85b6-4e37-8eec-19d81894140d\") "
Sep 08 20:30:49.134640 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.134614    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-bundle" (OuterVolumeSpecName: "bundle") pod "4c986b9f-85b6-4e37-8eec-19d81894140d" (UID: "4c986b9f-85b6-4e37-8eec-19d81894140d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:30:49.136069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.136036    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c986b9f-85b6-4e37-8eec-19d81894140d-kube-api-access-hvtrx" (OuterVolumeSpecName: "kube-api-access-hvtrx") pod "4c986b9f-85b6-4e37-8eec-19d81894140d" (UID: "4c986b9f-85b6-4e37-8eec-19d81894140d"). InnerVolumeSpecName "kube-api-access-hvtrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:30:49.139765 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.139738    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-util" (OuterVolumeSpecName: "util") pod "4c986b9f-85b6-4e37-8eec-19d81894140d" (UID: "4c986b9f-85b6-4e37-8eec-19d81894140d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:30:49.234671 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.234564    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvtrx\" (UniqueName: \"kubernetes.io/projected/4c986b9f-85b6-4e37-8eec-19d81894140d-kube-api-access-hvtrx\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:49.234671 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.234598    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-util\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:49.234671 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.234613    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c986b9f-85b6-4e37-8eec-19d81894140d-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:30:49.912298 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.912270    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t"
Sep 08 20:30:49.912298 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.912277    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7hj7t" event={"ID":"4c986b9f-85b6-4e37-8eec-19d81894140d","Type":"ContainerDied","Data":"7cfcea31dc5cbda928c3ccc8a95dd015793ac6b616bff402821406fab0c3a3e4"}
Sep 08 20:30:49.912498 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:30:49.912330    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cfcea31dc5cbda928c3ccc8a95dd015793ac6b616bff402821406fab0c3a3e4"
Sep 08 20:31:00.748434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.748399    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"]
Sep 08 20:31:00.748947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.748850    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerName="util"
Sep 08 20:31:00.748947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.748869    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerName="util"
Sep 08 20:31:00.748947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.748909    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerName="pull"
Sep 08 20:31:00.748947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.748918    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerName="pull"
Sep 08 20:31:00.748947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.748937    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerName="extract"
Sep 08 20:31:00.748947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.748945    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerName="extract"
Sep 08 20:31:00.749260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.749044    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c986b9f-85b6-4e37-8eec-19d81894140d" containerName="extract"
Sep 08 20:31:00.754846 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.754824    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.757257 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.757233    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 20:31:00.757368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.757272    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 20:31:00.757368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.757275    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-49dhb\""
Sep 08 20:31:00.762498 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.762474    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"]
Sep 08 20:31:00.828786 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.828748    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.828997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.828807    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.828997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.828825    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfm5c\" (UniqueName: \"kubernetes.io/projected/27fe5498-3246-4e58-b04a-78cedc81b479-kube-api-access-xfm5c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.930169 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.930128    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.930343 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.930196    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.930343 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.930225    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfm5c\" (UniqueName: \"kubernetes.io/projected/27fe5498-3246-4e58-b04a-78cedc81b479-kube-api-access-xfm5c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.930545 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.930523    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.930588 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.930545    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:00.947451 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:00.947425    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfm5c\" (UniqueName: \"kubernetes.io/projected/27fe5498-3246-4e58-b04a-78cedc81b479-kube-api-access-xfm5c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:01.065102 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.065008    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:01.338441 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.338413    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"]
Sep 08 20:31:01.341000 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:31:01.340964    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27fe5498_3246_4e58_b04a_78cedc81b479.slice/crio-5a00318f23631781bad46622c86caeb524019674bc4431a93822a6d5de596ca7 WatchSource:0}: Error finding container 5a00318f23631781bad46622c86caeb524019674bc4431a93822a6d5de596ca7: Status 404 returned error can't find the container with id 5a00318f23631781bad46622c86caeb524019674bc4431a93822a6d5de596ca7
Sep 08 20:31:01.882042 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.882006    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"]
Sep 08 20:31:01.885366 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.885346    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:01.888861 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.888807    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 20:31:01.889094 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.888998    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 20:31:01.889094 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.889002    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 20:31:01.889260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.889245    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sv8zf\""
Sep 08 20:31:01.889335 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.889313    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 20:31:01.903465 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.903432    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"]
Sep 08 20:31:01.938860 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.938812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71846c51-60c3-4773-b2e9-6f44d404c6e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:01.939051 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.938875    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71846c51-60c3-4773-b2e9-6f44d404c6e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:01.939051 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.938963    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71846c51-60c3-4773-b2e9-6f44d404c6e2-tmp\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:01.939051 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.939009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71846c51-60c3-4773-b2e9-6f44d404c6e2-manifests\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:01.939217 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.939051    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vgg\" (UniqueName: \"kubernetes.io/projected/71846c51-60c3-4773-b2e9-6f44d404c6e2-kube-api-access-z2vgg\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:01.973225 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.973191    2561 generic.go:356] "Generic (PLEG): container finished" podID="27fe5498-3246-4e58-b04a-78cedc81b479" containerID="02a1e98764f0c4a640651f4a60b4ca903c7235b2021001cb0b72cf6c05590c25" exitCode=0
Sep 08 20:31:01.973381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.973256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr" event={"ID":"27fe5498-3246-4e58-b04a-78cedc81b479","Type":"ContainerDied","Data":"02a1e98764f0c4a640651f4a60b4ca903c7235b2021001cb0b72cf6c05590c25"}
Sep 08 20:31:01.973381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:01.973303    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr" event={"ID":"27fe5498-3246-4e58-b04a-78cedc81b479","Type":"ContainerStarted","Data":"5a00318f23631781bad46622c86caeb524019674bc4431a93822a6d5de596ca7"}
Sep 08 20:31:02.039772 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.039742    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71846c51-60c3-4773-b2e9-6f44d404c6e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.039772 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.039780    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71846c51-60c3-4773-b2e9-6f44d404c6e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.040049 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.039813    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71846c51-60c3-4773-b2e9-6f44d404c6e2-tmp\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.040049 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.039852    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71846c51-60c3-4773-b2e9-6f44d404c6e2-manifests\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.040049 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.039914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vgg\" (UniqueName: \"kubernetes.io/projected/71846c51-60c3-4773-b2e9-6f44d404c6e2-kube-api-access-z2vgg\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.040353 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.040324    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71846c51-60c3-4773-b2e9-6f44d404c6e2-manifests\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.042309 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.042286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71846c51-60c3-4773-b2e9-6f44d404c6e2-tmp\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.042518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.042497    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71846c51-60c3-4773-b2e9-6f44d404c6e2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.042581 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.042549    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71846c51-60c3-4773-b2e9-6f44d404c6e2-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.051031 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.051009    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vgg\" (UniqueName: \"kubernetes.io/projected/71846c51-60c3-4773-b2e9-6f44d404c6e2-kube-api-access-z2vgg\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-bkbz5\" (UID: \"71846c51-60c3-4773-b2e9-6f44d404c6e2\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.195987 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.195952    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:02.472997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.472965    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"]
Sep 08 20:31:02.497383 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:31:02.497343    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71846c51_60c3_4773_b2e9_6f44d404c6e2.slice/crio-5a89dd03df5a065bdbb83068cde42babb5765bfd236345b1ba1bea742b04a980 WatchSource:0}: Error finding container 5a89dd03df5a065bdbb83068cde42babb5765bfd236345b1ba1bea742b04a980: Status 404 returned error can't find the container with id 5a89dd03df5a065bdbb83068cde42babb5765bfd236345b1ba1bea742b04a980
Sep 08 20:31:02.979673 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.979634    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5" event={"ID":"71846c51-60c3-4773-b2e9-6f44d404c6e2","Type":"ContainerStarted","Data":"5a89dd03df5a065bdbb83068cde42babb5765bfd236345b1ba1bea742b04a980"}
Sep 08 20:31:02.981524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.981496    2561 generic.go:356] "Generic (PLEG): container finished" podID="27fe5498-3246-4e58-b04a-78cedc81b479" containerID="5b384b93a548473e2e86694bc7f15ae8cb92eee18ae2817d4d7e3c291bdd7caf" exitCode=0
Sep 08 20:31:02.981645 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:02.981550    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr" event={"ID":"27fe5498-3246-4e58-b04a-78cedc81b479","Type":"ContainerDied","Data":"5b384b93a548473e2e86694bc7f15ae8cb92eee18ae2817d4d7e3c291bdd7caf"}
Sep 08 20:31:03.135706 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:31:03.135666    2561 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27fe5498_3246_4e58_b04a_78cedc81b479.slice/crio-c97ae747734fd7c6f9a97272637ecbe0a0a04e693b2b8fb3543cc52188f16c64.scope\": RecentStats: unable to find data in memory cache]"
Sep 08 20:31:03.991107 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:03.991067    2561 generic.go:356] "Generic (PLEG): container finished" podID="27fe5498-3246-4e58-b04a-78cedc81b479" containerID="c97ae747734fd7c6f9a97272637ecbe0a0a04e693b2b8fb3543cc52188f16c64" exitCode=0
Sep 08 20:31:03.991595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:03.991122    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr" event={"ID":"27fe5498-3246-4e58-b04a-78cedc81b479","Type":"ContainerDied","Data":"c97ae747734fd7c6f9a97272637ecbe0a0a04e693b2b8fb3543cc52188f16c64"}
Sep 08 20:31:05.637360 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.637329    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:05.674992 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.674963    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-util\") pod \"27fe5498-3246-4e58-b04a-78cedc81b479\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") "
Sep 08 20:31:05.675154 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.675018    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfm5c\" (UniqueName: \"kubernetes.io/projected/27fe5498-3246-4e58-b04a-78cedc81b479-kube-api-access-xfm5c\") pod \"27fe5498-3246-4e58-b04a-78cedc81b479\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") "
Sep 08 20:31:05.675154 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.675071    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-bundle\") pod \"27fe5498-3246-4e58-b04a-78cedc81b479\" (UID: \"27fe5498-3246-4e58-b04a-78cedc81b479\") "
Sep 08 20:31:05.675746 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.675717    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-bundle" (OuterVolumeSpecName: "bundle") pod "27fe5498-3246-4e58-b04a-78cedc81b479" (UID: "27fe5498-3246-4e58-b04a-78cedc81b479"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:31:05.677213 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.677179    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27fe5498-3246-4e58-b04a-78cedc81b479-kube-api-access-xfm5c" (OuterVolumeSpecName: "kube-api-access-xfm5c") pod "27fe5498-3246-4e58-b04a-78cedc81b479" (UID: "27fe5498-3246-4e58-b04a-78cedc81b479"). InnerVolumeSpecName "kube-api-access-xfm5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:31:05.681935 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.681910    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-util" (OuterVolumeSpecName: "util") pod "27fe5498-3246-4e58-b04a-78cedc81b479" (UID: "27fe5498-3246-4e58-b04a-78cedc81b479"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:31:05.776531 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.776494    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:05.776531 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.776524    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27fe5498-3246-4e58-b04a-78cedc81b479-util\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:05.776531 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:05.776534    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfm5c\" (UniqueName: \"kubernetes.io/projected/27fe5498-3246-4e58-b04a-78cedc81b479-kube-api-access-xfm5c\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:06.003704 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:06.003672    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5" event={"ID":"71846c51-60c3-4773-b2e9-6f44d404c6e2","Type":"ContainerStarted","Data":"9e6b19bcab520a63bfca061600d3924f4bebf2a51b999ee7945b8b8b076eccac"}
Sep 08 20:31:06.005693 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:06.005655    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr" event={"ID":"27fe5498-3246-4e58-b04a-78cedc81b479","Type":"ContainerDied","Data":"5a00318f23631781bad46622c86caeb524019674bc4431a93822a6d5de596ca7"}
Sep 08 20:31:06.005693 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:06.005678    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0czrdr"
Sep 08 20:31:06.005861 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:06.005701    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a00318f23631781bad46622c86caeb524019674bc4431a93822a6d5de596ca7"
Sep 08 20:31:07.012210 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:07.012173    2561 generic.go:356] "Generic (PLEG): container finished" podID="71846c51-60c3-4773-b2e9-6f44d404c6e2" containerID="9e6b19bcab520a63bfca061600d3924f4bebf2a51b999ee7945b8b8b076eccac" exitCode=0
Sep 08 20:31:07.012610 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:07.012231    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5" event={"ID":"71846c51-60c3-4773-b2e9-6f44d404c6e2","Type":"ContainerDied","Data":"9e6b19bcab520a63bfca061600d3924f4bebf2a51b999ee7945b8b8b076eccac"}
Sep 08 20:31:08.019165 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:08.019109    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5" event={"ID":"71846c51-60c3-4773-b2e9-6f44d404c6e2","Type":"ContainerStarted","Data":"6befe9e6da156e56f57658b8f19ba53cb7947f6db073d0014d8503feb78c8fb2"}
Sep 08 20:31:08.019595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:08.019293    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:08.047423 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:08.047372    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5" podStartSLOduration=2.404939147 podStartE2EDuration="7.047352839s" totalImagesPullingTime="3.548672028s" totalInitContainerRuntime="1.093741664s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:08.045429544 +0000 UTC m=+354.210600121" watchObservedRunningTime="2026-09-08 20:31:08.047352839 +0000 UTC m=+354.212523418"
Sep 08 20:31:19.027271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:19.027239    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-bkbz5"
Sep 08 20:31:24.115957 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.115922    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bf757d869-sddql"]
Sep 08 20:31:24.116356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.116237    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27fe5498-3246-4e58-b04a-78cedc81b479" containerName="util"
Sep 08 20:31:24.116356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.116247    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="27fe5498-3246-4e58-b04a-78cedc81b479" containerName="util"
Sep 08 20:31:24.116356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.116255    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27fe5498-3246-4e58-b04a-78cedc81b479" containerName="pull"
Sep 08 20:31:24.116356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.116261    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="27fe5498-3246-4e58-b04a-78cedc81b479" containerName="pull"
Sep 08 20:31:24.116356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.116274    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27fe5498-3246-4e58-b04a-78cedc81b479" containerName="extract"
Sep 08 20:31:24.116356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.116279    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="27fe5498-3246-4e58-b04a-78cedc81b479" containerName="extract"
Sep 08 20:31:24.116356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.116328    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="27fe5498-3246-4e58-b04a-78cedc81b479" containerName="extract"
Sep 08 20:31:24.125246 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.125220    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.128355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.128323    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-srbcb\""
Sep 08 20:31:24.128355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.128352    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 20:31:24.128670 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.128636    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 20:31:24.129658 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.129638    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bf757d869-sddql"]
Sep 08 20:31:24.230880 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.230846    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8t5n\" (UniqueName: \"kubernetes.io/projected/6b6950c4-add4-4969-8d1a-bfcbd268f02c-kube-api-access-n8t5n\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.231037 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.230951    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b6950c4-add4-4969-8d1a-bfcbd268f02c-tls-certs\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.231037 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.231008    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b6950c4-add4-4969-8d1a-bfcbd268f02c-tmp\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.331509 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.331469    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b6950c4-add4-4969-8d1a-bfcbd268f02c-tmp\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.331711 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.331517    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8t5n\" (UniqueName: \"kubernetes.io/projected/6b6950c4-add4-4969-8d1a-bfcbd268f02c-kube-api-access-n8t5n\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.331711 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.331594    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b6950c4-add4-4969-8d1a-bfcbd268f02c-tls-certs\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.333793 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.333768    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b6950c4-add4-4969-8d1a-bfcbd268f02c-tmp\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.334048 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.334028    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b6950c4-add4-4969-8d1a-bfcbd268f02c-tls-certs\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.340448 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.340422    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8t5n\" (UniqueName: \"kubernetes.io/projected/6b6950c4-add4-4969-8d1a-bfcbd268f02c-kube-api-access-n8t5n\") pod \"kube-auth-proxy-7bf757d869-sddql\" (UID: \"6b6950c4-add4-4969-8d1a-bfcbd268f02c\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.436814 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.436772    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql"
Sep 08 20:31:24.709480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:24.709445    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bf757d869-sddql"]
Sep 08 20:31:24.710801 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:31:24.710770    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b6950c4_add4_4969_8d1a_bfcbd268f02c.slice/crio-c2e54288a464b4c4cf0532bc327d0a339fa2f860ba02e12f389e2f1a3697bf91 WatchSource:0}: Error finding container c2e54288a464b4c4cf0532bc327d0a339fa2f860ba02e12f389e2f1a3697bf91: Status 404 returned error can't find the container with id c2e54288a464b4c4cf0532bc327d0a339fa2f860ba02e12f389e2f1a3697bf91
Sep 08 20:31:25.099129 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.099041    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql" event={"ID":"6b6950c4-add4-4969-8d1a-bfcbd268f02c","Type":"ContainerStarted","Data":"c2e54288a464b4c4cf0532bc327d0a339fa2f860ba02e12f389e2f1a3697bf91"}
Sep 08 20:31:25.276975 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.276937    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"]
Sep 08 20:31:25.281628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.281604    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.285136 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.285107    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 08 20:31:25.285250 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.285143    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 08 20:31:25.285295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.285144    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 08 20:31:25.286756 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.286738    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 20:31:25.287027 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.287008    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rctrw\""
Sep 08 20:31:25.295227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.295205    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"]
Sep 08 20:31:25.441921 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.441866    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.442205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.441960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.442205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.442002    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.442205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.442048    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.442205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.442131    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxdqm\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-kube-api-access-cxdqm\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.442205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.442170    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.442516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.442213    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/31a88d2e-9c7a-40ad-98c8-0361b63ca059-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.543504 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.543470    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.543678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.543550    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxdqm\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-kube-api-access-cxdqm\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.543678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.543600    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.543678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.543646    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/31a88d2e-9c7a-40ad-98c8-0361b63ca059-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.543849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.543686    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.543849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.543717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.543849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.543750    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.544664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.544592    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.546615 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.546570    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/31a88d2e-9c7a-40ad-98c8-0361b63ca059-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.546859 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.546839    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.546982 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.546857    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.547847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.547828    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.552162 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.552137    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.552839 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.552816    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxdqm\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-kube-api-access-cxdqm\") pod \"istiod-openshift-gateway-7b7995d5cd-mq4jp\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.591854 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.591785    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:25.889581 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:25.889482    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"]
Sep 08 20:31:25.894185 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:31:25.894152    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a88d2e_9c7a_40ad_98c8_0361b63ca059.slice/crio-a35d6d33af12346de523746e211c40a2a1d2578e6bd9094d963d817ebb7b7a1c WatchSource:0}: Error finding container a35d6d33af12346de523746e211c40a2a1d2578e6bd9094d963d817ebb7b7a1c: Status 404 returned error can't find the container with id a35d6d33af12346de523746e211c40a2a1d2578e6bd9094d963d817ebb7b7a1c
Sep 08 20:31:26.108434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:26.108346    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp" event={"ID":"31a88d2e-9c7a-40ad-98c8-0361b63ca059","Type":"ContainerStarted","Data":"a35d6d33af12346de523746e211c40a2a1d2578e6bd9094d963d817ebb7b7a1c"}
Sep 08 20:31:28.124097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:28.124018    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql" event={"ID":"6b6950c4-add4-4969-8d1a-bfcbd268f02c","Type":"ContainerStarted","Data":"0700071f6d2029968973fd465ce62072e77b36beede9f9faae98ce43fbd1f38a"}
Sep 08 20:31:28.142059 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:28.142001    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bf757d869-sddql" podStartSLOduration=1.133944643 podStartE2EDuration="4.14198103s" totalImagesPullingTime="3.008036387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:28.14027401 +0000 UTC m=+374.305444587" watchObservedRunningTime="2026-09-08 20:31:28.14198103 +0000 UTC m=+374.307151606"
Sep 08 20:31:29.485646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:29.485611    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:31:29.485874 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:29.485676    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:31:30.139849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:30.139807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp" event={"ID":"31a88d2e-9c7a-40ad-98c8-0361b63ca059","Type":"ContainerStarted","Data":"469917212a3f7d03b12fe721b9a058bcae6ab72fd3476ad3937a42b94dff9f6a"}
Sep 08 20:31:30.140071 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:30.139961    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:30.141673 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:30.141645    2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-mq4jp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 20:31:30.141800 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:30.141694    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp" podUID="31a88d2e-9c7a-40ad-98c8-0361b63ca059" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 20:31:30.163061 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:30.162995    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp" podStartSLOduration=1.5743024399999999 podStartE2EDuration="5.162971358s" totalImagesPullingTime="3.588668918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:30.158694375 +0000 UTC m=+376.323864965" watchObservedRunningTime="2026-09-08 20:31:30.162971358 +0000 UTC m=+376.328141937"
Sep 08 20:31:31.147000 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:31.146971    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:42.494766 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.494735    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"]
Sep 08 20:31:42.499567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.499527    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:42.503416 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.503374    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 08 20:31:42.503594 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.503457    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tpb6v\""
Sep 08 20:31:42.506482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.506457    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"]
Sep 08 20:31:42.508849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.508830    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"]
Sep 08 20:31:42.508980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.508936    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:42.511653 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.511630    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-v7hk9\""
Sep 08 20:31:42.512155 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.512137    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 08 20:31:42.520219 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.520162    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"]
Sep 08 20:31:42.600010 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.599968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bfdr\" (UniqueName: \"kubernetes.io/projected/5c899c43-da7b-4143-bd29-86ef5795a956-kube-api-access-7bfdr\") pod \"kserve-controller-manager-d6f4b65b8-4ljhv\" (UID: \"5c899c43-da7b-4143-bd29-86ef5795a956\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:42.600186 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.600027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c899c43-da7b-4143-bd29-86ef5795a956-cert\") pod \"kserve-controller-manager-d6f4b65b8-4ljhv\" (UID: \"5c899c43-da7b-4143-bd29-86ef5795a956\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:42.701484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.701443    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bd25186-ab2f-4cc8-b1b8-cad530122972-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pp25j\" (UID: \"0bd25186-ab2f-4cc8-b1b8-cad530122972\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:42.701484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.701490    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bfdr\" (UniqueName: \"kubernetes.io/projected/5c899c43-da7b-4143-bd29-86ef5795a956-kube-api-access-7bfdr\") pod \"kserve-controller-manager-d6f4b65b8-4ljhv\" (UID: \"5c899c43-da7b-4143-bd29-86ef5795a956\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:42.701694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.701534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c899c43-da7b-4143-bd29-86ef5795a956-cert\") pod \"kserve-controller-manager-d6f4b65b8-4ljhv\" (UID: \"5c899c43-da7b-4143-bd29-86ef5795a956\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:42.701694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.701561    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp6lw\" (UniqueName: \"kubernetes.io/projected/0bd25186-ab2f-4cc8-b1b8-cad530122972-kube-api-access-lp6lw\") pod \"llmisvc-controller-manager-777c6bd7d5-pp25j\" (UID: \"0bd25186-ab2f-4cc8-b1b8-cad530122972\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:42.701694 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:31:42.701654    2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 08 20:31:42.701790 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:31:42.701727    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c899c43-da7b-4143-bd29-86ef5795a956-cert podName:5c899c43-da7b-4143-bd29-86ef5795a956 nodeName:}" failed. No retries permitted until 2026-09-08 20:31:43.201710311 +0000 UTC m=+389.366880872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5c899c43-da7b-4143-bd29-86ef5795a956-cert") pod "kserve-controller-manager-d6f4b65b8-4ljhv" (UID: "5c899c43-da7b-4143-bd29-86ef5795a956") : secret "kserve-webhook-server-cert" not found
Sep 08 20:31:42.719404 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.719367    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bfdr\" (UniqueName: \"kubernetes.io/projected/5c899c43-da7b-4143-bd29-86ef5795a956-kube-api-access-7bfdr\") pod \"kserve-controller-manager-d6f4b65b8-4ljhv\" (UID: \"5c899c43-da7b-4143-bd29-86ef5795a956\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:42.802976 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.802873    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp6lw\" (UniqueName: \"kubernetes.io/projected/0bd25186-ab2f-4cc8-b1b8-cad530122972-kube-api-access-lp6lw\") pod \"llmisvc-controller-manager-777c6bd7d5-pp25j\" (UID: \"0bd25186-ab2f-4cc8-b1b8-cad530122972\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:42.802976 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.802967    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bd25186-ab2f-4cc8-b1b8-cad530122972-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pp25j\" (UID: \"0bd25186-ab2f-4cc8-b1b8-cad530122972\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:42.805445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.805419    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bd25186-ab2f-4cc8-b1b8-cad530122972-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pp25j\" (UID: \"0bd25186-ab2f-4cc8-b1b8-cad530122972\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:42.813589 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.813559    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp6lw\" (UniqueName: \"kubernetes.io/projected/0bd25186-ab2f-4cc8-b1b8-cad530122972-kube-api-access-lp6lw\") pod \"llmisvc-controller-manager-777c6bd7d5-pp25j\" (UID: \"0bd25186-ab2f-4cc8-b1b8-cad530122972\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:42.823484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:42.823461    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:43.102194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:43.102163    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"]
Sep 08 20:31:43.103717 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:31:43.103687    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0bd25186_ab2f_4cc8_b1b8_cad530122972.slice/crio-f84f3547dffc63516ff3214f5e05e8a8ce70744ba54c560df487f6ab90cddcde WatchSource:0}: Error finding container f84f3547dffc63516ff3214f5e05e8a8ce70744ba54c560df487f6ab90cddcde: Status 404 returned error can't find the container with id f84f3547dffc63516ff3214f5e05e8a8ce70744ba54c560df487f6ab90cddcde
Sep 08 20:31:43.205729 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:43.205691    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c899c43-da7b-4143-bd29-86ef5795a956-cert\") pod \"kserve-controller-manager-d6f4b65b8-4ljhv\" (UID: \"5c899c43-da7b-4143-bd29-86ef5795a956\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:43.208365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:43.208334    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c899c43-da7b-4143-bd29-86ef5795a956-cert\") pod \"kserve-controller-manager-d6f4b65b8-4ljhv\" (UID: \"5c899c43-da7b-4143-bd29-86ef5795a956\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:43.211888 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:43.211850    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j" event={"ID":"0bd25186-ab2f-4cc8-b1b8-cad530122972","Type":"ContainerStarted","Data":"f84f3547dffc63516ff3214f5e05e8a8ce70744ba54c560df487f6ab90cddcde"}
Sep 08 20:31:43.414783 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:43.414754    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:43.700694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:43.700567    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"]
Sep 08 20:31:43.703195 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:31:43.703169    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c899c43_da7b_4143_bd29_86ef5795a956.slice/crio-e943fa478ca7becbc9a55f388716bc8fe14540a8208a56bd4e70b62f60f899d7 WatchSource:0}: Error finding container e943fa478ca7becbc9a55f388716bc8fe14540a8208a56bd4e70b62f60f899d7: Status 404 returned error can't find the container with id e943fa478ca7becbc9a55f388716bc8fe14540a8208a56bd4e70b62f60f899d7
Sep 08 20:31:44.225218 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:44.225176    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv" event={"ID":"5c899c43-da7b-4143-bd29-86ef5795a956","Type":"ContainerStarted","Data":"e943fa478ca7becbc9a55f388716bc8fe14540a8208a56bd4e70b62f60f899d7"}
Sep 08 20:31:45.767530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:45.767493    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"]
Sep 08 20:31:45.767999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:45.767703    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp" podUID="31a88d2e-9c7a-40ad-98c8-0361b63ca059" containerName="discovery" containerID="cri-o://469917212a3f7d03b12fe721b9a058bcae6ab72fd3476ad3937a42b94dff9f6a" gracePeriod=30
Sep 08 20:31:46.147567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.147533    2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-mq4jp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.31:8080/ready\": dial tcp 10.132.0.31:8080: connect: connection refused" start-of-body=
Sep 08 20:31:46.147722 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.147597    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp" podUID="31a88d2e-9c7a-40ad-98c8-0361b63ca059" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.31:8080/ready\": dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 08 20:31:46.236483 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.236447    2561 generic.go:356] "Generic (PLEG): container finished" podID="31a88d2e-9c7a-40ad-98c8-0361b63ca059" containerID="469917212a3f7d03b12fe721b9a058bcae6ab72fd3476ad3937a42b94dff9f6a" exitCode=0
Sep 08 20:31:46.236651 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.236522    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp" event={"ID":"31a88d2e-9c7a-40ad-98c8-0361b63ca059","Type":"ContainerDied","Data":"469917212a3f7d03b12fe721b9a058bcae6ab72fd3476ad3937a42b94dff9f6a"}
Sep 08 20:31:46.670590 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.670567    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:46.748069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.748041    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-dns-cert\") pod \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") "
Sep 08 20:31:46.748203 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.748085    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-cacerts\") pod \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") "
Sep 08 20:31:46.748203 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.748120    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-kubeconfig\") pod \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") "
Sep 08 20:31:46.748203 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.748190    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-token\") pod \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") "
Sep 08 20:31:46.748379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.748255    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/31a88d2e-9c7a-40ad-98c8-0361b63ca059-local-certs\") pod \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") "
Sep 08 20:31:46.748379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.748307    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxdqm\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-kube-api-access-cxdqm\") pod \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") "
Sep 08 20:31:46.748379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.748353    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-ca-configmap\") pod \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\" (UID: \"31a88d2e-9c7a-40ad-98c8-0361b63ca059\") "
Sep 08 20:31:46.748813 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.748781    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "31a88d2e-9c7a-40ad-98c8-0361b63ca059" (UID: "31a88d2e-9c7a-40ad-98c8-0361b63ca059"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:31:46.750682 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.750656    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "31a88d2e-9c7a-40ad-98c8-0361b63ca059" (UID: "31a88d2e-9c7a-40ad-98c8-0361b63ca059"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:31:46.750998 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.750809    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-token" (OuterVolumeSpecName: "istio-token") pod "31a88d2e-9c7a-40ad-98c8-0361b63ca059" (UID: "31a88d2e-9c7a-40ad-98c8-0361b63ca059"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:31:46.750998 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.750818    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31a88d2e-9c7a-40ad-98c8-0361b63ca059-local-certs" (OuterVolumeSpecName: "local-certs") pod "31a88d2e-9c7a-40ad-98c8-0361b63ca059" (UID: "31a88d2e-9c7a-40ad-98c8-0361b63ca059"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:31:46.750998 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.750888    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-cacerts" (OuterVolumeSpecName: "cacerts") pod "31a88d2e-9c7a-40ad-98c8-0361b63ca059" (UID: "31a88d2e-9c7a-40ad-98c8-0361b63ca059"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:31:46.751224 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.751205    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "31a88d2e-9c7a-40ad-98c8-0361b63ca059" (UID: "31a88d2e-9c7a-40ad-98c8-0361b63ca059"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:31:46.751458 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.751441    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-kube-api-access-cxdqm" (OuterVolumeSpecName: "kube-api-access-cxdqm") pod "31a88d2e-9c7a-40ad-98c8-0361b63ca059" (UID: "31a88d2e-9c7a-40ad-98c8-0361b63ca059"). InnerVolumeSpecName "kube-api-access-cxdqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:31:46.849655 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.849618    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-dns-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:46.849655 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.849651    2561 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-cacerts\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:46.849655 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.849662    2561 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-kubeconfig\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:46.849655 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.849671    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:46.850140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.849680    2561 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/31a88d2e-9c7a-40ad-98c8-0361b63ca059-local-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:46.850140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.849688    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxdqm\" (UniqueName: \"kubernetes.io/projected/31a88d2e-9c7a-40ad-98c8-0361b63ca059-kube-api-access-cxdqm\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:46.850140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:46.849697    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/31a88d2e-9c7a-40ad-98c8-0361b63ca059-istio-csr-ca-configmap\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:31:47.243492 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.243458    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv" event={"ID":"5c899c43-da7b-4143-bd29-86ef5795a956","Type":"ContainerStarted","Data":"c79e8472f570e322a87dcdc44f49af0635395e0d88279833dddb8d64532254e4"}
Sep 08 20:31:47.243717 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.243583    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:31:47.245004 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.244980    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"
Sep 08 20:31:47.245138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.245015    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp" event={"ID":"31a88d2e-9c7a-40ad-98c8-0361b63ca059","Type":"ContainerDied","Data":"a35d6d33af12346de523746e211c40a2a1d2578e6bd9094d963d817ebb7b7a1c"}
Sep 08 20:31:47.245138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.245060    2561 scope.go:117] "RemoveContainer" containerID="469917212a3f7d03b12fe721b9a058bcae6ab72fd3476ad3937a42b94dff9f6a"
Sep 08 20:31:47.247272 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.247246    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j" event={"ID":"0bd25186-ab2f-4cc8-b1b8-cad530122972","Type":"ContainerStarted","Data":"d9e33cf74a523f1dce548e764d373333308aeb5970327419046b0233235b3ef1"}
Sep 08 20:31:47.247426 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.247409    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:31:47.262317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.262271    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv" podStartSLOduration=2.283405367 podStartE2EDuration="5.262254619s" totalImagesPullingTime="2.978849252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:47.260129614 +0000 UTC m=+393.425300191" watchObservedRunningTime="2026-09-08 20:31:47.262254619 +0000 UTC m=+393.427425194"
Sep 08 20:31:47.276138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.276103    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"]
Sep 08 20:31:47.282131 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.282102    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mq4jp"]
Sep 08 20:31:47.296814 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:47.296761    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j" podStartSLOduration=1.7140486990000001 podStartE2EDuration="5.296745606s" totalImagesPullingTime="3.582696907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:47.295679083 +0000 UTC m=+393.460849660" watchObservedRunningTime="2026-09-08 20:31:47.296745606 +0000 UTC m=+393.461916181"
Sep 08 20:31:48.363406 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:48.363375    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31a88d2e-9c7a-40ad-98c8-0361b63ca059" path="/var/lib/kubelet/pods/31a88d2e-9c7a-40ad-98c8-0361b63ca059/volumes"
Sep 08 20:31:53.948040 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.948001    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c4c56559d-xbb8q"]
Sep 08 20:31:53.948584 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.948549    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a88d2e-9c7a-40ad-98c8-0361b63ca059" containerName="discovery"
Sep 08 20:31:53.948584 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.948570    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a88d2e-9c7a-40ad-98c8-0361b63ca059" containerName="discovery"
Sep 08 20:31:53.948737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.948722    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="31a88d2e-9c7a-40ad-98c8-0361b63ca059" containerName="discovery"
Sep 08 20:31:53.953331 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.953306    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:53.956219 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.956188    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 20:31:53.957367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.957322    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zlxsw\""
Sep 08 20:31:53.957367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.957359    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 20:31:53.957639 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.957507    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 20:31:53.957718 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.957667    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 20:31:53.957770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.957730    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 20:31:53.962647 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.962612    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 08 20:31:53.964509 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:53.964484    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4c56559d-xbb8q"]
Sep 08 20:31:54.010575 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.010538    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-config\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.010719 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.010590    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-trusted-ca-bundle\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.010719 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.010693    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-service-ca\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.010805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.010729    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-serving-cert\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.010805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.010799    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-oauth-serving-cert\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.010872 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.010824    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-oauth-config\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.010872 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.010840    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxj5t\" (UniqueName: \"kubernetes.io/projected/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-kube-api-access-nxj5t\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.111883 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.111847    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-trusted-ca-bundle\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.111911    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-service-ca\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.111945    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-serving-cert\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.111987    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-oauth-serving-cert\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.112021    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-oauth-config\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.112047    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxj5t\" (UniqueName: \"kubernetes.io/projected/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-kube-api-access-nxj5t\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.112094    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-config\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112752 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.112725    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-service-ca\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112863 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.112789    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-oauth-serving-cert\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.112959 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.112938    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-config\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.113039 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.113024    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-trusted-ca-bundle\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.114473 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.114446    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-oauth-config\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.114565 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.114530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-console-serving-cert\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.120980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.120956    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxj5t\" (UniqueName: \"kubernetes.io/projected/e9cbbb9e-677a-4e46-88c9-dd500e8dd92d-kube-api-access-nxj5t\") pod \"console-6c4c56559d-xbb8q\" (UID: \"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d\") " pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.266695 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.266597    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:31:54.543386 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:54.543357    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4c56559d-xbb8q"]
Sep 08 20:31:54.545656 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:31:54.545629    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9cbbb9e_677a_4e46_88c9_dd500e8dd92d.slice/crio-0e6d8a7f7fe4aee1ded8472b9048b04599f34669ceb129951796e4e1cbb2e3d5 WatchSource:0}: Error finding container 0e6d8a7f7fe4aee1ded8472b9048b04599f34669ceb129951796e4e1cbb2e3d5: Status 404 returned error can't find the container with id 0e6d8a7f7fe4aee1ded8472b9048b04599f34669ceb129951796e4e1cbb2e3d5
Sep 08 20:31:55.300126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:55.300091    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4c56559d-xbb8q" event={"ID":"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d","Type":"ContainerStarted","Data":"845ede7061434dcc96d5329d9b992e07176466c3bc300c7a66048538fe449a97"}
Sep 08 20:31:55.300126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:55.300143    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4c56559d-xbb8q" event={"ID":"e9cbbb9e-677a-4e46-88c9-dd500e8dd92d","Type":"ContainerStarted","Data":"0e6d8a7f7fe4aee1ded8472b9048b04599f34669ceb129951796e4e1cbb2e3d5"}
Sep 08 20:31:55.323267 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:31:55.323211    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c4c56559d-xbb8q" podStartSLOduration=2.3231955539999998 podStartE2EDuration="2.323195554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:55.320682994 +0000 UTC m=+401.485853569" watchObservedRunningTime="2026-09-08 20:31:55.323195554 +0000 UTC m=+401.488366131"
Sep 08 20:32:04.266907 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:04.266859    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:32:04.267361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:04.266961    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:32:04.271857 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:04.271828    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:32:04.349017 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:04.348988    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c4c56559d-xbb8q"
Sep 08 20:32:18.255524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:18.255491    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pp25j"
Sep 08 20:32:18.258557 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:18.258532    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4ljhv"
Sep 08 20:32:37.161755 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.161672    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"]
Sep 08 20:32:37.169850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.169827    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.172816 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.172788    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 20:32:37.172992 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.172795    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 20:32:37.173698 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.173673    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-49dhb\""
Sep 08 20:32:37.174081 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.174059    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"]
Sep 08 20:32:37.204111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.204070    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.204298 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.204156    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhbgg\" (UniqueName: \"kubernetes.io/projected/544b4241-fef5-483d-b692-08ca60bf0166-kube-api-access-vhbgg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.204298 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.204190    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.260204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.260166    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"]
Sep 08 20:32:37.263704 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.263687    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.271236 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.271097    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"]
Sep 08 20:32:37.305158 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.305118    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.305158 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.305156    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.305381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.305191    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.305381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.305217    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8t98\" (UniqueName: \"kubernetes.io/projected/a5e76cff-cd28-4cdc-b5ff-7937759561e4-kube-api-access-b8t98\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.305381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.305258    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.305505 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.305482    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhbgg\" (UniqueName: \"kubernetes.io/projected/544b4241-fef5-483d-b692-08ca60bf0166-kube-api-access-vhbgg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.305577 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.305560    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.305616 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.305565    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.313911 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.313865    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhbgg\" (UniqueName: \"kubernetes.io/projected/544b4241-fef5-483d-b692-08ca60bf0166-kube-api-access-vhbgg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.360177 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.360139    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"]
Sep 08 20:32:37.363910 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.363877    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.373724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.373697    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"]
Sep 08 20:32:37.406575 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.406533    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8t98\" (UniqueName: \"kubernetes.io/projected/a5e76cff-cd28-4cdc-b5ff-7937759561e4-kube-api-access-b8t98\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.406785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.406581    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.406785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.406611    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.406785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.406661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.406785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.406769    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxrpn\" (UniqueName: \"kubernetes.io/projected/5eb7b524-b331-4b1f-81e0-30f2a0948e06-kube-api-access-sxrpn\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.407101 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.406939    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.407211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.407191    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.407264 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.407221    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.415092 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.415029    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8t98\" (UniqueName: \"kubernetes.io/projected/a5e76cff-cd28-4cdc-b5ff-7937759561e4-kube-api-access-b8t98\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.461459 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.461424    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"]
Sep 08 20:32:37.465207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.465181    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.474263 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.474234    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"]
Sep 08 20:32:37.481325 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.481295    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:37.508204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.508164    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.508395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.508222    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.508395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.508254    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.508395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.508305    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djfxn\" (UniqueName: \"kubernetes.io/projected/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-kube-api-access-djfxn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.508395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.508386    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxrpn\" (UniqueName: \"kubernetes.io/projected/5eb7b524-b331-4b1f-81e0-30f2a0948e06-kube-api-access-sxrpn\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.508628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.508411    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.508628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.508572    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.508628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.508582    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.517149 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.517121    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxrpn\" (UniqueName: \"kubernetes.io/projected/5eb7b524-b331-4b1f-81e0-30f2a0948e06-kube-api-access-sxrpn\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.573636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.573595    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:37.609267 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.609226    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.609463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.609321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.609463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.609369    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djfxn\" (UniqueName: \"kubernetes.io/projected/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-kube-api-access-djfxn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.609803 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.609692    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.609949 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.609882    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.618737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.618705    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djfxn\" (UniqueName: \"kubernetes.io/projected/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-kube-api-access-djfxn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.674391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.674304    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:37.759060 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.759030    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"]
Sep 08 20:32:37.761223 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:32:37.761197    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod544b4241_fef5_483d_b692_08ca60bf0166.slice/crio-ca591fe2fd33a6e8568a4bb0b0f0ef926532d8acdb733309228904c9ab0d6d26 WatchSource:0}: Error finding container ca591fe2fd33a6e8568a4bb0b0f0ef926532d8acdb733309228904c9ab0d6d26: Status 404 returned error can't find the container with id ca591fe2fd33a6e8568a4bb0b0f0ef926532d8acdb733309228904c9ab0d6d26
Sep 08 20:32:37.775753 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.775720    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:37.862026 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.861988    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"]
Sep 08 20:32:37.865431 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:32:37.865400    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e76cff_cd28_4cdc_b5ff_7937759561e4.slice/crio-6687ac9a8c7bce85e837dfbae7fc339dd5e491ec817e8693333b493892792307 WatchSource:0}: Error finding container 6687ac9a8c7bce85e837dfbae7fc339dd5e491ec817e8693333b493892792307: Status 404 returned error can't find the container with id 6687ac9a8c7bce85e837dfbae7fc339dd5e491ec817e8693333b493892792307
Sep 08 20:32:37.948332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:37.948310    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"]
Sep 08 20:32:37.959133 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:32:37.959104    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb7b524_b331_4b1f_81e0_30f2a0948e06.slice/crio-ba7e96f68997ba4601045e586ad56c085c83b0149b6eaab0ca8d2163096f9117 WatchSource:0}: Error finding container ba7e96f68997ba4601045e586ad56c085c83b0149b6eaab0ca8d2163096f9117: Status 404 returned error can't find the container with id ba7e96f68997ba4601045e586ad56c085c83b0149b6eaab0ca8d2163096f9117
Sep 08 20:32:38.054987 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.054950    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"]
Sep 08 20:32:38.115789 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:32:38.115752    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66f047e_6b10_4f1c_b1d0_715c4d225fcc.slice/crio-e4527d7a990304a9ee70a59394cd5b95a6db7dc52cbe271425aa91e896b77195 WatchSource:0}: Error finding container e4527d7a990304a9ee70a59394cd5b95a6db7dc52cbe271425aa91e896b77195: Status 404 returned error can't find the container with id e4527d7a990304a9ee70a59394cd5b95a6db7dc52cbe271425aa91e896b77195
Sep 08 20:32:38.514261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.514160    2561 generic.go:356] "Generic (PLEG): container finished" podID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerID="4f3c0f39fe592dc1a86e60b8564ea6ceb4cf0a8edf9e79d06c84cf6559bd3238" exitCode=0
Sep 08 20:32:38.514261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.514254    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2" event={"ID":"a5e76cff-cd28-4cdc-b5ff-7937759561e4","Type":"ContainerDied","Data":"4f3c0f39fe592dc1a86e60b8564ea6ceb4cf0a8edf9e79d06c84cf6559bd3238"}
Sep 08 20:32:38.514758 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.514308    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2" event={"ID":"a5e76cff-cd28-4cdc-b5ff-7937759561e4","Type":"ContainerStarted","Data":"6687ac9a8c7bce85e837dfbae7fc339dd5e491ec817e8693333b493892792307"}
Sep 08 20:32:38.515906 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.515861    2561 generic.go:356] "Generic (PLEG): container finished" podID="544b4241-fef5-483d-b692-08ca60bf0166" containerID="c075fbae6a1e6afba0093986ee1cd45c5a139678d7c83dcf915a47083dd2a494" exitCode=0
Sep 08 20:32:38.515986 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.515930    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr" event={"ID":"544b4241-fef5-483d-b692-08ca60bf0166","Type":"ContainerDied","Data":"c075fbae6a1e6afba0093986ee1cd45c5a139678d7c83dcf915a47083dd2a494"}
Sep 08 20:32:38.515986 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.515973    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr" event={"ID":"544b4241-fef5-483d-b692-08ca60bf0166","Type":"ContainerStarted","Data":"ca591fe2fd33a6e8568a4bb0b0f0ef926532d8acdb733309228904c9ab0d6d26"}
Sep 08 20:32:38.518088 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.517907    2561 generic.go:356] "Generic (PLEG): container finished" podID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerID="956e9395c058e4bdbffa64b967a0c0d5f0a3e7f930bd0aec3bea0efd64cb094f" exitCode=0
Sep 08 20:32:38.518088 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.517997    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24" event={"ID":"5eb7b524-b331-4b1f-81e0-30f2a0948e06","Type":"ContainerDied","Data":"956e9395c058e4bdbffa64b967a0c0d5f0a3e7f930bd0aec3bea0efd64cb094f"}
Sep 08 20:32:38.518088 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.518044    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24" event={"ID":"5eb7b524-b331-4b1f-81e0-30f2a0948e06","Type":"ContainerStarted","Data":"ba7e96f68997ba4601045e586ad56c085c83b0149b6eaab0ca8d2163096f9117"}
Sep 08 20:32:38.519970 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.519945    2561 generic.go:356] "Generic (PLEG): container finished" podID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerID="c499c4d0d4fe9944ffb875558b0aae1318b4211a7f197069a053d7bdb39818bd" exitCode=0
Sep 08 20:32:38.520078 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.520020    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn" event={"ID":"d66f047e-6b10-4f1c-b1d0-715c4d225fcc","Type":"ContainerDied","Data":"c499c4d0d4fe9944ffb875558b0aae1318b4211a7f197069a053d7bdb39818bd"}
Sep 08 20:32:38.520126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:38.520080    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn" event={"ID":"d66f047e-6b10-4f1c-b1d0-715c4d225fcc","Type":"ContainerStarted","Data":"e4527d7a990304a9ee70a59394cd5b95a6db7dc52cbe271425aa91e896b77195"}
Sep 08 20:32:40.532294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:40.532253    2561 generic.go:356] "Generic (PLEG): container finished" podID="544b4241-fef5-483d-b692-08ca60bf0166" containerID="e9325aee532a8c78a8d70a62d531efe3cae8214b4ce84ae21ca039f0fd291226" exitCode=0
Sep 08 20:32:40.532744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:40.532307    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr" event={"ID":"544b4241-fef5-483d-b692-08ca60bf0166","Type":"ContainerDied","Data":"e9325aee532a8c78a8d70a62d531efe3cae8214b4ce84ae21ca039f0fd291226"}
Sep 08 20:32:40.534374 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:40.534353    2561 generic.go:356] "Generic (PLEG): container finished" podID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerID="710eec5d911ef19a5ab5ea9f8157ec5182e4f07ac9a219cd4777e7883a85beb9" exitCode=0
Sep 08 20:32:40.534475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:40.534433    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24" event={"ID":"5eb7b524-b331-4b1f-81e0-30f2a0948e06","Type":"ContainerDied","Data":"710eec5d911ef19a5ab5ea9f8157ec5182e4f07ac9a219cd4777e7883a85beb9"}
Sep 08 20:32:40.536495 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:40.536474    2561 generic.go:356] "Generic (PLEG): container finished" podID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerID="681c26103f1ecc507863a90a000d9a8d59b7c982ea7d60379865a75fb7cef6b7" exitCode=0
Sep 08 20:32:40.536588 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:40.536517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn" event={"ID":"d66f047e-6b10-4f1c-b1d0-715c4d225fcc","Type":"ContainerDied","Data":"681c26103f1ecc507863a90a000d9a8d59b7c982ea7d60379865a75fb7cef6b7"}
Sep 08 20:32:40.538677 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:40.538655    2561 generic.go:356] "Generic (PLEG): container finished" podID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerID="460ea5a62b470b4d4bfb4763c3d33ec5fa7a17f029dc33d74f2ca9cf00a825b5" exitCode=0
Sep 08 20:32:40.538767 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:40.538711    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2" event={"ID":"a5e76cff-cd28-4cdc-b5ff-7937759561e4","Type":"ContainerDied","Data":"460ea5a62b470b4d4bfb4763c3d33ec5fa7a17f029dc33d74f2ca9cf00a825b5"}
Sep 08 20:32:41.547055 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:41.547019    2561 generic.go:356] "Generic (PLEG): container finished" podID="544b4241-fef5-483d-b692-08ca60bf0166" containerID="d4b599fdbcc42123304f52eb1bf5dcebc6624a7dede6c47b4014fda3154bc811" exitCode=0
Sep 08 20:32:41.547501 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:41.547109    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr" event={"ID":"544b4241-fef5-483d-b692-08ca60bf0166","Type":"ContainerDied","Data":"d4b599fdbcc42123304f52eb1bf5dcebc6624a7dede6c47b4014fda3154bc811"}
Sep 08 20:32:41.549305 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:41.549280    2561 generic.go:356] "Generic (PLEG): container finished" podID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerID="58cf2df444dcb77d038912af11c0a1b5f3ca42c6d163389e2f54d3404a8228cf" exitCode=0
Sep 08 20:32:41.549442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:41.549364    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24" event={"ID":"5eb7b524-b331-4b1f-81e0-30f2a0948e06","Type":"ContainerDied","Data":"58cf2df444dcb77d038912af11c0a1b5f3ca42c6d163389e2f54d3404a8228cf"}
Sep 08 20:32:41.551489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:41.551468    2561 generic.go:356] "Generic (PLEG): container finished" podID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerID="e8aff6269abc952ddbb2dde7847f539f4235cc58b895aa495d71c581f2364f29" exitCode=0
Sep 08 20:32:41.551590 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:41.551535    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn" event={"ID":"d66f047e-6b10-4f1c-b1d0-715c4d225fcc","Type":"ContainerDied","Data":"e8aff6269abc952ddbb2dde7847f539f4235cc58b895aa495d71c581f2364f29"}
Sep 08 20:32:41.553534 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:41.553514    2561 generic.go:356] "Generic (PLEG): container finished" podID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerID="73886b3e19a986fb85336f216fc1e45b2f7b41ae71941f28b7ed671afe005181" exitCode=0
Sep 08 20:32:41.553638 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:41.553562    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2" event={"ID":"a5e76cff-cd28-4cdc-b5ff-7937759561e4","Type":"ContainerDied","Data":"73886b3e19a986fb85336f216fc1e45b2f7b41ae71941f28b7ed671afe005181"}
Sep 08 20:32:42.748195 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.748139    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:42.751312 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.751285    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:42.755844 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.755816    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:42.764686 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.764634    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:42.857579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857481    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8t98\" (UniqueName: \"kubernetes.io/projected/a5e76cff-cd28-4cdc-b5ff-7937759561e4-kube-api-access-b8t98\") pod \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") "
Sep 08 20:32:42.857579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857537    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-util\") pod \"544b4241-fef5-483d-b692-08ca60bf0166\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") "
Sep 08 20:32:42.857579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857567    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-util\") pod \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") "
Sep 08 20:32:42.857841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857630    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-bundle\") pod \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\" (UID: \"a5e76cff-cd28-4cdc-b5ff-7937759561e4\") "
Sep 08 20:32:42.857841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857659    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-bundle\") pod \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") "
Sep 08 20:32:42.857841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857686    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-util\") pod \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") "
Sep 08 20:32:42.857841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857714    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhbgg\" (UniqueName: \"kubernetes.io/projected/544b4241-fef5-483d-b692-08ca60bf0166-kube-api-access-vhbgg\") pod \"544b4241-fef5-483d-b692-08ca60bf0166\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") "
Sep 08 20:32:42.857841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857757    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-bundle\") pod \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") "
Sep 08 20:32:42.857841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857789    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-util\") pod \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") "
Sep 08 20:32:42.857841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857838    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djfxn\" (UniqueName: \"kubernetes.io/projected/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-kube-api-access-djfxn\") pod \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\" (UID: \"d66f047e-6b10-4f1c-b1d0-715c4d225fcc\") "
Sep 08 20:32:42.858208 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857933    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-bundle\") pod \"544b4241-fef5-483d-b692-08ca60bf0166\" (UID: \"544b4241-fef5-483d-b692-08ca60bf0166\") "
Sep 08 20:32:42.858208 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.857958    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxrpn\" (UniqueName: \"kubernetes.io/projected/5eb7b524-b331-4b1f-81e0-30f2a0948e06-kube-api-access-sxrpn\") pod \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\" (UID: \"5eb7b524-b331-4b1f-81e0-30f2a0948e06\") "
Sep 08 20:32:42.858368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.858341    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-bundle" (OuterVolumeSpecName: "bundle") pod "5eb7b524-b331-4b1f-81e0-30f2a0948e06" (UID: "5eb7b524-b331-4b1f-81e0-30f2a0948e06"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:32:42.858437 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.858378    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-bundle" (OuterVolumeSpecName: "bundle") pod "a5e76cff-cd28-4cdc-b5ff-7937759561e4" (UID: "a5e76cff-cd28-4cdc-b5ff-7937759561e4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:32:42.858836 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.858809    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-bundle" (OuterVolumeSpecName: "bundle") pod "d66f047e-6b10-4f1c-b1d0-715c4d225fcc" (UID: "d66f047e-6b10-4f1c-b1d0-715c4d225fcc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:32:42.859793 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.859725    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.859793 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.859752    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.859793 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.859766    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.860088 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.860063    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-bundle" (OuterVolumeSpecName: "bundle") pod "544b4241-fef5-483d-b692-08ca60bf0166" (UID: "544b4241-fef5-483d-b692-08ca60bf0166"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:32:42.860919 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.860869    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5e76cff-cd28-4cdc-b5ff-7937759561e4-kube-api-access-b8t98" (OuterVolumeSpecName: "kube-api-access-b8t98") pod "a5e76cff-cd28-4cdc-b5ff-7937759561e4" (UID: "a5e76cff-cd28-4cdc-b5ff-7937759561e4"). InnerVolumeSpecName "kube-api-access-b8t98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:32:42.861006 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.860926    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-kube-api-access-djfxn" (OuterVolumeSpecName: "kube-api-access-djfxn") pod "d66f047e-6b10-4f1c-b1d0-715c4d225fcc" (UID: "d66f047e-6b10-4f1c-b1d0-715c4d225fcc"). InnerVolumeSpecName "kube-api-access-djfxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:32:42.861798 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.861771    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/544b4241-fef5-483d-b692-08ca60bf0166-kube-api-access-vhbgg" (OuterVolumeSpecName: "kube-api-access-vhbgg") pod "544b4241-fef5-483d-b692-08ca60bf0166" (UID: "544b4241-fef5-483d-b692-08ca60bf0166"). InnerVolumeSpecName "kube-api-access-vhbgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:32:42.861936 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.861918    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb7b524-b331-4b1f-81e0-30f2a0948e06-kube-api-access-sxrpn" (OuterVolumeSpecName: "kube-api-access-sxrpn") pod "5eb7b524-b331-4b1f-81e0-30f2a0948e06" (UID: "5eb7b524-b331-4b1f-81e0-30f2a0948e06"). InnerVolumeSpecName "kube-api-access-sxrpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:32:42.865027 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.864993    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-util" (OuterVolumeSpecName: "util") pod "d66f047e-6b10-4f1c-b1d0-715c4d225fcc" (UID: "d66f047e-6b10-4f1c-b1d0-715c4d225fcc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:32:42.865270 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.865251    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-util" (OuterVolumeSpecName: "util") pod "544b4241-fef5-483d-b692-08ca60bf0166" (UID: "544b4241-fef5-483d-b692-08ca60bf0166"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:32:42.867155 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.867132    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-util" (OuterVolumeSpecName: "util") pod "a5e76cff-cd28-4cdc-b5ff-7937759561e4" (UID: "a5e76cff-cd28-4cdc-b5ff-7937759561e4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:32:42.867202 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.867176    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-util" (OuterVolumeSpecName: "util") pod "5eb7b524-b331-4b1f-81e0-30f2a0948e06" (UID: "5eb7b524-b331-4b1f-81e0-30f2a0948e06"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:32:42.960697 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960657    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-util\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.960697 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960688    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djfxn\" (UniqueName: \"kubernetes.io/projected/d66f047e-6b10-4f1c-b1d0-715c4d225fcc-kube-api-access-djfxn\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.960697 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960700    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-bundle\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.960697 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960710    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxrpn\" (UniqueName: \"kubernetes.io/projected/5eb7b524-b331-4b1f-81e0-30f2a0948e06-kube-api-access-sxrpn\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.960999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960718    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8t98\" (UniqueName: \"kubernetes.io/projected/a5e76cff-cd28-4cdc-b5ff-7937759561e4-kube-api-access-b8t98\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.960999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960726    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/544b4241-fef5-483d-b692-08ca60bf0166-util\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.960999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960734    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e76cff-cd28-4cdc-b5ff-7937759561e4-util\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.960999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960741    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eb7b524-b331-4b1f-81e0-30f2a0948e06-util\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:42.960999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:42.960749    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhbgg\" (UniqueName: \"kubernetes.io/projected/544b4241-fef5-483d-b692-08ca60bf0166-kube-api-access-vhbgg\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:32:43.566805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.566777    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr"
Sep 08 20:32:43.566805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.566782    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b4lgr" event={"ID":"544b4241-fef5-483d-b692-08ca60bf0166","Type":"ContainerDied","Data":"ca591fe2fd33a6e8568a4bb0b0f0ef926532d8acdb733309228904c9ab0d6d26"}
Sep 08 20:32:43.567085 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.566831    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca591fe2fd33a6e8568a4bb0b0f0ef926532d8acdb733309228904c9ab0d6d26"
Sep 08 20:32:43.568704 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.568684    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24"
Sep 08 20:32:43.568814 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.568688    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xkm24" event={"ID":"5eb7b524-b331-4b1f-81e0-30f2a0948e06","Type":"ContainerDied","Data":"ba7e96f68997ba4601045e586ad56c085c83b0149b6eaab0ca8d2163096f9117"}
Sep 08 20:32:43.568814 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.568803    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba7e96f68997ba4601045e586ad56c085c83b0149b6eaab0ca8d2163096f9117"
Sep 08 20:32:43.570634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.570610    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn" event={"ID":"d66f047e-6b10-4f1c-b1d0-715c4d225fcc","Type":"ContainerDied","Data":"e4527d7a990304a9ee70a59394cd5b95a6db7dc52cbe271425aa91e896b77195"}
Sep 08 20:32:43.570634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.570633    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5rrwn"
Sep 08 20:32:43.570802 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.570652    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4527d7a990304a9ee70a59394cd5b95a6db7dc52cbe271425aa91e896b77195"
Sep 08 20:32:43.572623 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.572602    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2" event={"ID":"a5e76cff-cd28-4cdc-b5ff-7937759561e4","Type":"ContainerDied","Data":"6687ac9a8c7bce85e837dfbae7fc339dd5e491ec817e8693333b493892792307"}
Sep 08 20:32:43.572723 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.572634    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6687ac9a8c7bce85e837dfbae7fc339dd5e491ec817e8693333b493892792307"
Sep 08 20:32:43.572723 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:43.572636    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ad6rd2"
Sep 08 20:32:51.884263 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884224    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"]
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884621    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="544b4241-fef5-483d-b692-08ca60bf0166" containerName="util"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884633    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="544b4241-fef5-483d-b692-08ca60bf0166" containerName="util"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884639    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerName="pull"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884644    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerName="pull"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884650    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerName="util"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884655    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerName="util"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884662    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="544b4241-fef5-483d-b692-08ca60bf0166" containerName="pull"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884667    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="544b4241-fef5-483d-b692-08ca60bf0166" containerName="pull"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884678    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerName="extract"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884683    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerName="extract"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884688    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerName="extract"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884693    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerName="extract"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884702    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerName="pull"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884707    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerName="pull"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884713    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerName="pull"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884723    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerName="pull"
Sep 08 20:32:51.884724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884730    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="544b4241-fef5-483d-b692-08ca60bf0166" containerName="extract"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884735    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="544b4241-fef5-483d-b692-08ca60bf0166" containerName="extract"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884740    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerName="extract"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884745    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerName="extract"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884751    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerName="util"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884755    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerName="util"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884765    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerName="util"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884770    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerName="util"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884830    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5e76cff-cd28-4cdc-b5ff-7937759561e4" containerName="extract"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884839    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="544b4241-fef5-483d-b692-08ca60bf0166" containerName="extract"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884845    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb7b524-b331-4b1f-81e0-30f2a0948e06" containerName="extract"
Sep 08 20:32:51.885254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.884851    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d66f047e-6b10-4f1c-b1d0-715c4d225fcc" containerName="extract"
Sep 08 20:32:51.889442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.889420    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:51.892235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.892210    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 20:32:51.892342 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.892277    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xfsfx\""
Sep 08 20:32:51.892342 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.892279    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 20:32:51.904368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:51.904334    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"]
Sep 08 20:32:52.038137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.038101    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05b4b998-04d0-4582-a0c3-36b8c0835959-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" (UID: \"05b4b998-04d0-4582-a0c3-36b8c0835959\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:52.038328 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.038159    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fpbb\" (UniqueName: \"kubernetes.io/projected/05b4b998-04d0-4582-a0c3-36b8c0835959-kube-api-access-5fpbb\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" (UID: \"05b4b998-04d0-4582-a0c3-36b8c0835959\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:52.139463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.139368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fpbb\" (UniqueName: \"kubernetes.io/projected/05b4b998-04d0-4582-a0c3-36b8c0835959-kube-api-access-5fpbb\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" (UID: \"05b4b998-04d0-4582-a0c3-36b8c0835959\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:52.139624 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.139495    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05b4b998-04d0-4582-a0c3-36b8c0835959-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" (UID: \"05b4b998-04d0-4582-a0c3-36b8c0835959\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:52.139938 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.139889    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05b4b998-04d0-4582-a0c3-36b8c0835959-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" (UID: \"05b4b998-04d0-4582-a0c3-36b8c0835959\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:52.165730 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.165693    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fpbb\" (UniqueName: \"kubernetes.io/projected/05b4b998-04d0-4582-a0c3-36b8c0835959-kube-api-access-5fpbb\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" (UID: \"05b4b998-04d0-4582-a0c3-36b8c0835959\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:52.199852 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.199813    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:52.486023 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.485997    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"]
Sep 08 20:32:52.488092 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:32:52.488062    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b4b998_04d0_4582_a0c3_36b8c0835959.slice/crio-92c82cd02a596d798f45e1b90b12a9d26de9367a1b4d7745ec3f5725b75965d2 WatchSource:0}: Error finding container 92c82cd02a596d798f45e1b90b12a9d26de9367a1b4d7745ec3f5725b75965d2: Status 404 returned error can't find the container with id 92c82cd02a596d798f45e1b90b12a9d26de9367a1b4d7745ec3f5725b75965d2
Sep 08 20:32:52.624258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:52.624218    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99" event={"ID":"05b4b998-04d0-4582-a0c3-36b8c0835959","Type":"ContainerStarted","Data":"92c82cd02a596d798f45e1b90b12a9d26de9367a1b4d7745ec3f5725b75965d2"}
Sep 08 20:32:54.622801 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.622759    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"]
Sep 08 20:32:54.634120 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.634082    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"
Sep 08 20:32:54.638770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.638700    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dtcr4\""
Sep 08 20:32:54.639683 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.639562    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 08 20:32:54.640699 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.640670    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"]
Sep 08 20:32:54.766813 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.766768    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hc2s\" (UniqueName: \"kubernetes.io/projected/09b25262-3a59-4ad5-8767-b30f3198e5c5-kube-api-access-2hc2s\") pod \"dns-operator-controller-manager-7b84cd7c9c-tqflh\" (UID: \"09b25262-3a59-4ad5-8767-b30f3198e5c5\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"
Sep 08 20:32:54.868284 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.868246    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hc2s\" (UniqueName: \"kubernetes.io/projected/09b25262-3a59-4ad5-8767-b30f3198e5c5-kube-api-access-2hc2s\") pod \"dns-operator-controller-manager-7b84cd7c9c-tqflh\" (UID: \"09b25262-3a59-4ad5-8767-b30f3198e5c5\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"
Sep 08 20:32:54.877709 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.877638    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hc2s\" (UniqueName: \"kubernetes.io/projected/09b25262-3a59-4ad5-8767-b30f3198e5c5-kube-api-access-2hc2s\") pod \"dns-operator-controller-manager-7b84cd7c9c-tqflh\" (UID: \"09b25262-3a59-4ad5-8767-b30f3198e5c5\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"
Sep 08 20:32:54.949389 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:54.949351    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"
Sep 08 20:32:55.235666 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:55.235639    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"]
Sep 08 20:32:56.534132 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:32:56.534093    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b25262_3a59_4ad5_8767_b30f3198e5c5.slice/crio-cda7cc254147e63c65af785b823cc7eb802c506d00d7693e6741bb276dfe9978 WatchSource:0}: Error finding container cda7cc254147e63c65af785b823cc7eb802c506d00d7693e6741bb276dfe9978: Status 404 returned error can't find the container with id cda7cc254147e63c65af785b823cc7eb802c506d00d7693e6741bb276dfe9978
Sep 08 20:32:56.654303 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:56.654269    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh" event={"ID":"09b25262-3a59-4ad5-8767-b30f3198e5c5","Type":"ContainerStarted","Data":"cda7cc254147e63c65af785b823cc7eb802c506d00d7693e6741bb276dfe9978"}
Sep 08 20:32:57.663442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:57.663398    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99" event={"ID":"05b4b998-04d0-4582-a0c3-36b8c0835959","Type":"ContainerStarted","Data":"e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670"}
Sep 08 20:32:57.663881 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:57.663466    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:32:57.686965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:57.686907    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99" podStartSLOduration=2.568323399 podStartE2EDuration="6.686875103s" totalImagesPullingTime="4.118551704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:32:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:32:57.683582281 +0000 UTC m=+463.848752856" watchObservedRunningTime="2026-09-08 20:32:57.686875103 +0000 UTC m=+463.852045681"
Sep 08 20:32:58.299875 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.299835    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q8879"]
Sep 08 20:32:58.303380 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.303358    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-q8879"
Sep 08 20:32:58.305830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.305810    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kbqqd\""
Sep 08 20:32:58.315655 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.315627    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q8879"]
Sep 08 20:32:58.403142 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.403108    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bksz7\" (UniqueName: \"kubernetes.io/projected/50eb3405-5e3c-44b5-b81b-34afbacf7bbf-kube-api-access-bksz7\") pod \"authorino-operator-76b478bd78-q8879\" (UID: \"50eb3405-5e3c-44b5-b81b-34afbacf7bbf\") " pod="kuadrant-system/authorino-operator-76b478bd78-q8879"
Sep 08 20:32:58.504402 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.504311    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bksz7\" (UniqueName: \"kubernetes.io/projected/50eb3405-5e3c-44b5-b81b-34afbacf7bbf-kube-api-access-bksz7\") pod \"authorino-operator-76b478bd78-q8879\" (UID: \"50eb3405-5e3c-44b5-b81b-34afbacf7bbf\") " pod="kuadrant-system/authorino-operator-76b478bd78-q8879"
Sep 08 20:32:58.514508 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.514475    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bksz7\" (UniqueName: \"kubernetes.io/projected/50eb3405-5e3c-44b5-b81b-34afbacf7bbf-kube-api-access-bksz7\") pod \"authorino-operator-76b478bd78-q8879\" (UID: \"50eb3405-5e3c-44b5-b81b-34afbacf7bbf\") " pod="kuadrant-system/authorino-operator-76b478bd78-q8879"
Sep 08 20:32:58.614045 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.614007    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-q8879"
Sep 08 20:32:58.673867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.673830    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh" event={"ID":"09b25262-3a59-4ad5-8767-b30f3198e5c5","Type":"ContainerStarted","Data":"407681c4089b695473c697d81f2095c168794ca432609ddde40639e5fb847059"}
Sep 08 20:32:58.674256 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.674182    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"
Sep 08 20:32:58.692727 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.692668    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh" podStartSLOduration=3.070439098 podStartE2EDuration="4.692650715s" totalImagesPullingTime="1.622211617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:32:58.69210477 +0000 UTC m=+464.857275344" watchObservedRunningTime="2026-09-08 20:32:58.692650715 +0000 UTC m=+464.857821291"
Sep 08 20:32:58.900070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:58.900046    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q8879"]
Sep 08 20:32:58.902369 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:32:58.902340    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50eb3405_5e3c_44b5_b81b_34afbacf7bbf.slice/crio-76c03eb5034750f50590e0a171507531ef5a00d4ad2e62afe2afae1634d737f4 WatchSource:0}: Error finding container 76c03eb5034750f50590e0a171507531ef5a00d4ad2e62afe2afae1634d737f4: Status 404 returned error can't find the container with id 76c03eb5034750f50590e0a171507531ef5a00d4ad2e62afe2afae1634d737f4
Sep 08 20:32:59.611776 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.611741    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"]
Sep 08 20:32:59.616888 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.616863    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.619223 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.619200    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j2hc8\""
Sep 08 20:32:59.619356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.619317    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 08 20:32:59.619474 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.619459    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 08 20:32:59.626424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.626404    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"]
Sep 08 20:32:59.681810 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.681780    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-q8879" event={"ID":"50eb3405-5e3c-44b5-b81b-34afbacf7bbf","Type":"ContainerStarted","Data":"76c03eb5034750f50590e0a171507531ef5a00d4ad2e62afe2afae1634d737f4"}
Sep 08 20:32:59.716080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.716044    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ed40483-8bef-4048-aec9-722a9869e33c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.716080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.716082    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ed40483-8bef-4048-aec9-722a9869e33c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.716293 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.716164    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rckgx\" (UniqueName: \"kubernetes.io/projected/1ed40483-8bef-4048-aec9-722a9869e33c-kube-api-access-rckgx\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.817610 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.817567    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rckgx\" (UniqueName: \"kubernetes.io/projected/1ed40483-8bef-4048-aec9-722a9869e33c-kube-api-access-rckgx\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.817790 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.817670    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ed40483-8bef-4048-aec9-722a9869e33c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.817790 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.817694    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ed40483-8bef-4048-aec9-722a9869e33c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.818402 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.818375    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ed40483-8bef-4048-aec9-722a9869e33c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.820144 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.820127    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ed40483-8bef-4048-aec9-722a9869e33c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.825926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.825890    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rckgx\" (UniqueName: \"kubernetes.io/projected/1ed40483-8bef-4048-aec9-722a9869e33c-kube-api-access-rckgx\") pod \"kuadrant-console-plugin-fbdd4f445-tnlpb\" (UID: \"1ed40483-8bef-4048-aec9-722a9869e33c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:32:59.928615 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:32:59.928585    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"
Sep 08 20:33:00.201849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:00.201818    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb"]
Sep 08 20:33:00.203540 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:33:00.203516    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ed40483_8bef_4048_aec9_722a9869e33c.slice/crio-d35ffd084c77870ff34dde6e14f26c27daaa41a294d79a2e77461fb58e875926 WatchSource:0}: Error finding container d35ffd084c77870ff34dde6e14f26c27daaa41a294d79a2e77461fb58e875926: Status 404 returned error can't find the container with id d35ffd084c77870ff34dde6e14f26c27daaa41a294d79a2e77461fb58e875926
Sep 08 20:33:00.691373 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:00.691339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb" event={"ID":"1ed40483-8bef-4048-aec9-722a9869e33c","Type":"ContainerStarted","Data":"d35ffd084c77870ff34dde6e14f26c27daaa41a294d79a2e77461fb58e875926"}
Sep 08 20:33:01.708181 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:01.708149    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"]
Sep 08 20:33:01.711886 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:01.711856    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:01.714579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:01.714552    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7w87l\""
Sep 08 20:33:01.720093 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:01.720067    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"]
Sep 08 20:33:01.837810 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:01.837771    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgpj9\" (UniqueName: \"kubernetes.io/projected/a35efc35-3549-4026-aedd-3f61655d6b0a-kube-api-access-rgpj9\") pod \"limitador-operator-controller-manager-74495c69f8-zgx6n\" (UID: \"a35efc35-3549-4026-aedd-3f61655d6b0a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:01.938415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:01.938369    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgpj9\" (UniqueName: \"kubernetes.io/projected/a35efc35-3549-4026-aedd-3f61655d6b0a-kube-api-access-rgpj9\") pod \"limitador-operator-controller-manager-74495c69f8-zgx6n\" (UID: \"a35efc35-3549-4026-aedd-3f61655d6b0a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:01.947169 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:01.947125    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgpj9\" (UniqueName: \"kubernetes.io/projected/a35efc35-3549-4026-aedd-3f61655d6b0a-kube-api-access-rgpj9\") pod \"limitador-operator-controller-manager-74495c69f8-zgx6n\" (UID: \"a35efc35-3549-4026-aedd-3f61655d6b0a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:02.027631 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:02.027535    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:02.334112 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:02.334079    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"]
Sep 08 20:33:02.494299 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:33:02.494270    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35efc35_3549_4026_aedd_3f61655d6b0a.slice/crio-789fef80ace51c5cadc292bb9ba84f2b06f5cd8a13d901167482a0c893216e33 WatchSource:0}: Error finding container 789fef80ace51c5cadc292bb9ba84f2b06f5cd8a13d901167482a0c893216e33: Status 404 returned error can't find the container with id 789fef80ace51c5cadc292bb9ba84f2b06f5cd8a13d901167482a0c893216e33
Sep 08 20:33:02.715846 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:02.715808    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n" event={"ID":"a35efc35-3549-4026-aedd-3f61655d6b0a","Type":"ContainerStarted","Data":"789fef80ace51c5cadc292bb9ba84f2b06f5cd8a13d901167482a0c893216e33"}
Sep 08 20:33:05.737860 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:05.737756    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n" event={"ID":"a35efc35-3549-4026-aedd-3f61655d6b0a","Type":"ContainerStarted","Data":"1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3"}
Sep 08 20:33:05.738343 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:05.737885    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:05.739989 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:05.739951    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb" event={"ID":"1ed40483-8bef-4048-aec9-722a9869e33c","Type":"ContainerStarted","Data":"53ce31312bb3d6b9293b45c91ce218ca83b500969689393f50525d0763c7910e"}
Sep 08 20:33:05.742311 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:05.742256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-q8879" event={"ID":"50eb3405-5e3c-44b5-b81b-34afbacf7bbf","Type":"ContainerStarted","Data":"ffca4211a13db9172361eaf6a32e84e047c3387c3b5c19cedebc97be70a839af"}
Sep 08 20:33:05.742440 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:05.742416    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-q8879"
Sep 08 20:33:05.755317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:05.755258    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n" podStartSLOduration=1.8068096919999999 podStartE2EDuration="4.755238537s" totalImagesPullingTime="2.948428845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:05.754819838 +0000 UTC m=+471.919990415" watchObservedRunningTime="2026-09-08 20:33:05.755238537 +0000 UTC m=+471.920409117"
Sep 08 20:33:05.779384 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:05.779336    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-tnlpb" podStartSLOduration=1.538824187 podStartE2EDuration="6.779320342s" totalImagesPullingTime="5.240496155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:05.777130715 +0000 UTC m=+471.942301291" watchObservedRunningTime="2026-09-08 20:33:05.779320342 +0000 UTC m=+471.944490918"
Sep 08 20:33:05.795600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:05.795551    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-q8879" podStartSLOduration=1.270346629 podStartE2EDuration="7.795534631s" totalImagesPullingTime="6.525188002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:05.795451148 +0000 UTC m=+471.960621724" watchObservedRunningTime="2026-09-08 20:33:05.795534631 +0000 UTC m=+471.960705208"
Sep 08 20:33:08.676828 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:08.676796    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:09.684795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:09.684763    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-tqflh"
Sep 08 20:33:10.938507 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.934427    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"]
Sep 08 20:33:10.938507 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.934751    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" containerName="manager" containerID="cri-o://e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670" gracePeriod=2
Sep 08 20:33:10.945623 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.945588    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"]
Sep 08 20:33:10.958195 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.958164    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"]
Sep 08 20:33:10.958424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.958351    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" containerName="manager" containerID="cri-o://1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3" gracePeriod=2
Sep 08 20:33:10.960298 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.960275    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:10.969871 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:10.969835    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:10.970109 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.970066    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"]
Sep 08 20:33:10.970611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.970587    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" containerName="manager"
Sep 08 20:33:10.970611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.970613    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" containerName="manager"
Sep 08 20:33:10.970776 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.970720    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" containerName="manager"
Sep 08 20:33:10.972000 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:10.971963    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:10.973993 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:10.973960    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:10.974241 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.974225    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"]
Sep 08 20:33:10.974323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.974315    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:10.976484 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:10.976461    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:10.982462 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.982435    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"]
Sep 08 20:33:10.983020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.983005    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" containerName="manager"
Sep 08 20:33:10.983072 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.983024    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" containerName="manager"
Sep 08 20:33:10.983116 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.983106    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" containerName="manager"
Sep 08 20:33:10.986484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.986464    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"
Sep 08 20:33:10.988725 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:10.988699    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"]
Sep 08 20:33:11.000037 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.000009    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"]
Sep 08 20:33:11.007872 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.007834    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.023033 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.023003    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7be0efdf-5e13-4a0f-8564-586dc3ba707c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-w8jdp\" (UID: \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:11.023188 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.023043    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwsp4\" (UniqueName: \"kubernetes.io/projected/19e6dd31-06df-4f05-83b2-4efe18f97aef-kube-api-access-vwsp4\") pod \"limitador-operator-controller-manager-74495c69f8-jl79h\" (UID: \"19e6dd31-06df-4f05-83b2-4efe18f97aef\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"
Sep 08 20:33:11.023270 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.023187    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmp22\" (UniqueName: \"kubernetes.io/projected/7be0efdf-5e13-4a0f-8564-586dc3ba707c-kube-api-access-zmp22\") pod \"kuadrant-operator-controller-manager-76f8f686bf-w8jdp\" (UID: \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:11.028646 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.028613    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.030931 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.030879    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.124017 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.123977    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7be0efdf-5e13-4a0f-8564-586dc3ba707c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-w8jdp\" (UID: \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:11.124180 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.124030    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwsp4\" (UniqueName: \"kubernetes.io/projected/19e6dd31-06df-4f05-83b2-4efe18f97aef-kube-api-access-vwsp4\") pod \"limitador-operator-controller-manager-74495c69f8-jl79h\" (UID: \"19e6dd31-06df-4f05-83b2-4efe18f97aef\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"
Sep 08 20:33:11.124180 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.124143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmp22\" (UniqueName: \"kubernetes.io/projected/7be0efdf-5e13-4a0f-8564-586dc3ba707c-kube-api-access-zmp22\") pod \"kuadrant-operator-controller-manager-76f8f686bf-w8jdp\" (UID: \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:11.124843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.124820    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7be0efdf-5e13-4a0f-8564-586dc3ba707c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-w8jdp\" (UID: \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:11.135855 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.135757    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwsp4\" (UniqueName: \"kubernetes.io/projected/19e6dd31-06df-4f05-83b2-4efe18f97aef-kube-api-access-vwsp4\") pod \"limitador-operator-controller-manager-74495c69f8-jl79h\" (UID: \"19e6dd31-06df-4f05-83b2-4efe18f97aef\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"
Sep 08 20:33:11.138049 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.138025    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmp22\" (UniqueName: \"kubernetes.io/projected/7be0efdf-5e13-4a0f-8564-586dc3ba707c-kube-api-access-zmp22\") pod \"kuadrant-operator-controller-manager-76f8f686bf-w8jdp\" (UID: \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:11.196433 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.196410    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.198827 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.198799    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.200272 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.200256    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.200673 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.200651    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.202655 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.202637    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.204454 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.204435    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.224842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.224817    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05b4b998-04d0-4582-a0c3-36b8c0835959-extensions-socket-volume\") pod \"05b4b998-04d0-4582-a0c3-36b8c0835959\" (UID: \"05b4b998-04d0-4582-a0c3-36b8c0835959\") "
Sep 08 20:33:11.224935 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.224913    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fpbb\" (UniqueName: \"kubernetes.io/projected/05b4b998-04d0-4582-a0c3-36b8c0835959-kube-api-access-5fpbb\") pod \"05b4b998-04d0-4582-a0c3-36b8c0835959\" (UID: \"05b4b998-04d0-4582-a0c3-36b8c0835959\") "
Sep 08 20:33:11.225014 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.224997    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgpj9\" (UniqueName: \"kubernetes.io/projected/a35efc35-3549-4026-aedd-3f61655d6b0a-kube-api-access-rgpj9\") pod \"a35efc35-3549-4026-aedd-3f61655d6b0a\" (UID: \"a35efc35-3549-4026-aedd-3f61655d6b0a\") "
Sep 08 20:33:11.225333 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.225309    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05b4b998-04d0-4582-a0c3-36b8c0835959-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "05b4b998-04d0-4582-a0c3-36b8c0835959" (UID: "05b4b998-04d0-4582-a0c3-36b8c0835959"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:33:11.227242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.227217    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a35efc35-3549-4026-aedd-3f61655d6b0a-kube-api-access-rgpj9" (OuterVolumeSpecName: "kube-api-access-rgpj9") pod "a35efc35-3549-4026-aedd-3f61655d6b0a" (UID: "a35efc35-3549-4026-aedd-3f61655d6b0a"). InnerVolumeSpecName "kube-api-access-rgpj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:33:11.227242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.227234    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05b4b998-04d0-4582-a0c3-36b8c0835959-kube-api-access-5fpbb" (OuterVolumeSpecName: "kube-api-access-5fpbb") pod "05b4b998-04d0-4582-a0c3-36b8c0835959" (UID: "05b4b998-04d0-4582-a0c3-36b8c0835959"). InnerVolumeSpecName "kube-api-access-5fpbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:33:11.326205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.326169    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05b4b998-04d0-4582-a0c3-36b8c0835959-extensions-socket-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:33:11.326205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.326200    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fpbb\" (UniqueName: \"kubernetes.io/projected/05b4b998-04d0-4582-a0c3-36b8c0835959-kube-api-access-5fpbb\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:33:11.326205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.326211    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgpj9\" (UniqueName: \"kubernetes.io/projected/a35efc35-3549-4026-aedd-3f61655d6b0a-kube-api-access-rgpj9\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:33:11.379306 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.379270    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:11.385258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.385237    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"
Sep 08 20:33:11.676851 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:33:11.676823    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be0efdf_5e13_4a0f_8564_586dc3ba707c.slice/crio-f0e02c9f9e91648e89b719850808262818882c4d839a5f7cfdfb2a075054b12b WatchSource:0}: Error finding container f0e02c9f9e91648e89b719850808262818882c4d839a5f7cfdfb2a075054b12b: Status 404 returned error can't find the container with id f0e02c9f9e91648e89b719850808262818882c4d839a5f7cfdfb2a075054b12b
Sep 08 20:33:11.682999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.682975    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"]
Sep 08 20:33:11.683947 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:33:11.683921    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e6dd31_06df_4f05_83b2_4efe18f97aef.slice/crio-1937613fa7bcc1b77a00c6653b71a56732cdccf264d305977f84764776ac9f9f WatchSource:0}: Error finding container 1937613fa7bcc1b77a00c6653b71a56732cdccf264d305977f84764776ac9f9f: Status 404 returned error can't find the container with id 1937613fa7bcc1b77a00c6653b71a56732cdccf264d305977f84764776ac9f9f
Sep 08 20:33:11.685599 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.685576    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"]
Sep 08 20:33:11.778029 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.777998    2561 generic.go:356] "Generic (PLEG): container finished" podID="a35efc35-3549-4026-aedd-3f61655d6b0a" containerID="1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3" exitCode=0
Sep 08 20:33:11.778325 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.778045    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.778325 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.778106    2561 scope.go:117] "RemoveContainer" containerID="1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3"
Sep 08 20:33:11.780039 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.780010    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h" event={"ID":"19e6dd31-06df-4f05-83b2-4efe18f97aef","Type":"ContainerStarted","Data":"16117348c8a7811aac51442fc0e748aeed6c057ef9221688eca0decae813349a"}
Sep 08 20:33:11.780165 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.780069    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h" event={"ID":"19e6dd31-06df-4f05-83b2-4efe18f97aef","Type":"ContainerStarted","Data":"1937613fa7bcc1b77a00c6653b71a56732cdccf264d305977f84764776ac9f9f"}
Sep 08 20:33:11.780165 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.780126    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"
Sep 08 20:33:11.781867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.781501    2561 generic.go:356] "Generic (PLEG): container finished" podID="05b4b998-04d0-4582-a0c3-36b8c0835959" containerID="e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670" exitCode=0
Sep 08 20:33:11.781867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.781563    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.782237 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.782208    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.783841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.783820    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp" event={"ID":"7be0efdf-5e13-4a0f-8564-586dc3ba707c","Type":"ContainerStarted","Data":"52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0"}
Sep 08 20:33:11.783947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.783862    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp" event={"ID":"7be0efdf-5e13-4a0f-8564-586dc3ba707c","Type":"ContainerStarted","Data":"f0e02c9f9e91648e89b719850808262818882c4d839a5f7cfdfb2a075054b12b"}
Sep 08 20:33:11.783947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.783885    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:11.784344 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.784320    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.786566 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.786541    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.788922 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.788809    2561 scope.go:117] "RemoveContainer" containerID="1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3"
Sep 08 20:33:11.789388 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.789365    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3\": container with ID starting with 1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3 not found: ID does not exist" containerID="1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3"
Sep 08 20:33:11.789465 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.789396    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3"} err="failed to get container status \"1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3\": rpc error: code = NotFound desc = could not find container \"1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3\": container with ID starting with 1e7da30682ea0245cf8f5b44fc8d09533322a8ff2acaaab9cb1f9460487449e3 not found: ID does not exist"
Sep 08 20:33:11.789465 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.789414    2561 scope.go:117] "RemoveContainer" containerID="e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670"
Sep 08 20:33:11.797983 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.797967    2561 scope.go:117] "RemoveContainer" containerID="e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670"
Sep 08 20:33:11.798264 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.798247    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670\": container with ID starting with e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670 not found: ID does not exist" containerID="e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670"
Sep 08 20:33:11.798314 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.798270    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670"} err="failed to get container status \"e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670\": rpc error: code = NotFound desc = could not find container \"e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670\": container with ID starting with e962c58ac8662789007ccdf299fdac9b2ca682206d609a756d4774aa81930670 not found: ID does not exist"
Sep 08 20:33:11.823269 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.823218    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h" podStartSLOduration=1.823201947 podStartE2EDuration="1.823201947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:11.823179103 +0000 UTC m=+477.988349680" watchObservedRunningTime="2026-09-08 20:33:11.823201947 +0000 UTC m=+477.988372524"
Sep 08 20:33:11.826462 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.826429    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.856999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.856945    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp" podStartSLOduration=1.856927726 podStartE2EDuration="1.856927726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:11.855371937 +0000 UTC m=+478.020542512" watchObservedRunningTime="2026-09-08 20:33:11.856927726 +0000 UTC m=+478.022098301"
Sep 08 20:33:11.859104 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.859073    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:11.861660 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.861626    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.978002 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.977972    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"]
Sep 08 20:33:11.982681 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.982664    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:11.984993 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:11.984963    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8n99\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8n99"
Sep 08 20:33:11.997553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:11.997530    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"]
Sep 08 20:33:12.019405 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:12.019370    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zgx6n\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zgx6n"
Sep 08 20:33:12.032564 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.032537    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8f85\" (UniqueName: \"kubernetes.io/projected/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-kube-api-access-v8f85\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tb5tr\" (UID: \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:12.032720 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.032639    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tb5tr\" (UID: \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:12.134156 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.134115    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8f85\" (UniqueName: \"kubernetes.io/projected/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-kube-api-access-v8f85\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tb5tr\" (UID: \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:12.134341 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.134228    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tb5tr\" (UID: \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:12.134686 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.134661    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tb5tr\" (UID: \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:12.143024 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.142997    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8f85\" (UniqueName: \"kubernetes.io/projected/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-kube-api-access-v8f85\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tb5tr\" (UID: \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:12.293417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.293312    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:12.364309 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.364274    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05b4b998-04d0-4582-a0c3-36b8c0835959" path="/var/lib/kubelet/pods/05b4b998-04d0-4582-a0c3-36b8c0835959/volumes"
Sep 08 20:33:12.364596 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.364582    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a35efc35-3549-4026-aedd-3f61655d6b0a" path="/var/lib/kubelet/pods/a35efc35-3549-4026-aedd-3f61655d6b0a/volumes"
Sep 08 20:33:12.571263 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.571223    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"]
Sep 08 20:33:12.573153 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:33:12.573123    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e6f58d_3f17_4cc5_8164_06ed8d20b962.slice/crio-b111d430704f4368618f4dec17919f092bc4a849d18f590e45211c71843287b4 WatchSource:0}: Error finding container b111d430704f4368618f4dec17919f092bc4a849d18f590e45211c71843287b4: Status 404 returned error can't find the container with id b111d430704f4368618f4dec17919f092bc4a849d18f590e45211c71843287b4
Sep 08 20:33:12.793858 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.793823    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr" event={"ID":"d2e6f58d-3f17-4cc5-8164-06ed8d20b962","Type":"ContainerStarted","Data":"941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91"}
Sep 08 20:33:12.794025 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.793880    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr" event={"ID":"d2e6f58d-3f17-4cc5-8164-06ed8d20b962","Type":"ContainerStarted","Data":"b111d430704f4368618f4dec17919f092bc4a849d18f590e45211c71843287b4"}
Sep 08 20:33:12.794126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.794108    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:12.816618 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:12.816564    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr" podStartSLOduration=1.8165476900000002 podStartE2EDuration="1.81654769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:33:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:12.813887682 +0000 UTC m=+478.979058260" watchObservedRunningTime="2026-09-08 20:33:12.81654769 +0000 UTC m=+478.981718293"
Sep 08 20:33:16.751105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:16.751070    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-q8879"
Sep 08 20:33:22.796064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:22.796035    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jl79h"
Sep 08 20:33:22.796452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:22.796112    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:23.802317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:23.802286    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:33:23.864572 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:23.864523    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"]
Sep 08 20:33:23.864822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:23.864780    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp" podUID="7be0efdf-5e13-4a0f-8564-586dc3ba707c" containerName="manager" containerID="cri-o://52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0" gracePeriod=10
Sep 08 20:33:24.128658 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.128627    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:24.241831 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.241796    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7be0efdf-5e13-4a0f-8564-586dc3ba707c-extensions-socket-volume\") pod \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\" (UID: \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\") "
Sep 08 20:33:24.242072 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.241867    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmp22\" (UniqueName: \"kubernetes.io/projected/7be0efdf-5e13-4a0f-8564-586dc3ba707c-kube-api-access-zmp22\") pod \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\" (UID: \"7be0efdf-5e13-4a0f-8564-586dc3ba707c\") "
Sep 08 20:33:24.242216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.242192    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be0efdf-5e13-4a0f-8564-586dc3ba707c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7be0efdf-5e13-4a0f-8564-586dc3ba707c" (UID: "7be0efdf-5e13-4a0f-8564-586dc3ba707c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:33:24.243928 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.243907    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be0efdf-5e13-4a0f-8564-586dc3ba707c-kube-api-access-zmp22" (OuterVolumeSpecName: "kube-api-access-zmp22") pod "7be0efdf-5e13-4a0f-8564-586dc3ba707c" (UID: "7be0efdf-5e13-4a0f-8564-586dc3ba707c"). InnerVolumeSpecName "kube-api-access-zmp22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:33:24.343670 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.343575    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmp22\" (UniqueName: \"kubernetes.io/projected/7be0efdf-5e13-4a0f-8564-586dc3ba707c-kube-api-access-zmp22\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:33:24.343670 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.343612    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7be0efdf-5e13-4a0f-8564-586dc3ba707c-extensions-socket-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:33:24.869516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.869480    2561 generic.go:356] "Generic (PLEG): container finished" podID="7be0efdf-5e13-4a0f-8564-586dc3ba707c" containerID="52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0" exitCode=0
Sep 08 20:33:24.870030 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.869529    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"
Sep 08 20:33:24.870030 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.869562    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp" event={"ID":"7be0efdf-5e13-4a0f-8564-586dc3ba707c","Type":"ContainerDied","Data":"52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0"}
Sep 08 20:33:24.870030 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.869617    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp" event={"ID":"7be0efdf-5e13-4a0f-8564-586dc3ba707c","Type":"ContainerDied","Data":"f0e02c9f9e91648e89b719850808262818882c4d839a5f7cfdfb2a075054b12b"}
Sep 08 20:33:24.870030 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.869643    2561 scope.go:117] "RemoveContainer" containerID="52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0"
Sep 08 20:33:24.878583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.878567    2561 scope.go:117] "RemoveContainer" containerID="52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0"
Sep 08 20:33:24.878836 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:33:24.878813    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0\": container with ID starting with 52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0 not found: ID does not exist" containerID="52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0"
Sep 08 20:33:24.878915 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.878848    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0"} err="failed to get container status \"52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0\": rpc error: code = NotFound desc = could not find container \"52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0\": container with ID starting with 52cdf0b75d37d7b3db4be43e6fec3c4767c8c9c3744bd2bf1d37f044d44665f0 not found: ID does not exist"
Sep 08 20:33:24.892234 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.892209    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"]
Sep 08 20:33:24.899828 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:24.899791    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-w8jdp"]
Sep 08 20:33:26.363839 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:26.363804    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be0efdf-5e13-4a0f-8564-586dc3ba707c" path="/var/lib/kubelet/pods/7be0efdf-5e13-4a0f-8564-586dc3ba707c/volumes"
Sep 08 20:33:40.186450 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.186396    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"]
Sep 08 20:33:40.187009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.186875    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be0efdf-5e13-4a0f-8564-586dc3ba707c" containerName="manager"
Sep 08 20:33:40.187009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.186889    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be0efdf-5e13-4a0f-8564-586dc3ba707c" containerName="manager"
Sep 08 20:33:40.187009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.186970    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be0efdf-5e13-4a0f-8564-586dc3ba707c" containerName="manager"
Sep 08 20:33:40.190100 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.190077    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.192822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.192798    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 20:33:40.193023 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.193000    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mmhm7\""
Sep 08 20:33:40.203878 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.203841    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"]
Sep 08 20:33:40.294853 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.294812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.294853 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.294850    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.295123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.294871    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.295123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.295014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.295123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.295059    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.295244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.295122    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.295244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.295177    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xgh4\" (UniqueName: \"kubernetes.io/projected/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-kube-api-access-9xgh4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.295244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.295199    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.295356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.295254    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396284 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396250    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396466 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396309    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xgh4\" (UniqueName: \"kubernetes.io/projected/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-kube-api-access-9xgh4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396466 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396347    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396466 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396397    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396466 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396429    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396466 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396454    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396563    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.396977 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396952    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.397071 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.396991    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.397071 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.397006    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.397222 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.397194    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.397388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.397367    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.399102 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.399080    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.399192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.399177    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.404200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.404169    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.404364 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.404263    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xgh4\" (UniqueName: \"kubernetes.io/projected/ddd858c1-029d-4d85-ac36-dce67c4f4ebf-kube-api-access-9xgh4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kv8wj\" (UID: \"ddd858c1-029d-4d85-ac36-dce67c4f4ebf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.504723 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.504619    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:40.782874 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.782840    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"]
Sep 08 20:33:40.784214 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:33:40.784176    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd858c1_029d_4d85_ac36_dce67c4f4ebf.slice/crio-497d4c0db7016a1ca32224d7756906aa3536afd9919c8c3c22ecb60f8601bd2c WatchSource:0}: Error finding container 497d4c0db7016a1ca32224d7756906aa3536afd9919c8c3c22ecb60f8601bd2c: Status 404 returned error can't find the container with id 497d4c0db7016a1ca32224d7756906aa3536afd9919c8c3c22ecb60f8601bd2c
Sep 08 20:33:40.965674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:40.965634    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" event={"ID":"ddd858c1-029d-4d85-ac36-dce67c4f4ebf","Type":"ContainerStarted","Data":"497d4c0db7016a1ca32224d7756906aa3536afd9919c8c3c22ecb60f8601bd2c"}
Sep 08 20:33:43.481073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:43.481031    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:33:43.481469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:43.481126    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:33:43.481469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:43.481174    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:33:43.986724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:43.986692    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" event={"ID":"ddd858c1-029d-4d85-ac36-dce67c4f4ebf","Type":"ContainerStarted","Data":"67ddcda28d151205047aea658311774fb8ab642ffaae5113fc9ddff2510659da"}
Sep 08 20:33:44.008602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:44.008551    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" podStartSLOduration=1.314088258 podStartE2EDuration="4.008535361s" totalImagesPullingTime="2.694447103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:33:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:44.006191964 +0000 UTC m=+510.171362539" watchObservedRunningTime="2026-09-08 20:33:44.008535361 +0000 UTC m=+510.173705937"
Sep 08 20:33:44.505626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:44.505586    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:44.510704 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:44.510677    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:44.992387 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:44.992360    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:44.993290 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:44.993270    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:33:45.034980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.034946    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-fsbjh"]
Sep 08 20:33:45.038932 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.038880    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:45.041575 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.041548    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 08 20:33:45.051525 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.051499    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-fsbjh"]
Sep 08 20:33:45.128490 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.128454    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-config-file\") pod \"limitador-limitador-8588749977-fsbjh\" (UID: \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\") " pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:45.128674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.128515    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq27k\" (UniqueName: \"kubernetes.io/projected/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-kube-api-access-sq27k\") pod \"limitador-limitador-8588749977-fsbjh\" (UID: \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\") " pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:45.132676 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.132646    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-fsbjh"]
Sep 08 20:33:45.230706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.229696    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-config-file\") pod \"limitador-limitador-8588749977-fsbjh\" (UID: \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\") " pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:45.230706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.229779    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq27k\" (UniqueName: \"kubernetes.io/projected/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-kube-api-access-sq27k\") pod \"limitador-limitador-8588749977-fsbjh\" (UID: \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\") " pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:45.231122 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.231095    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-config-file\") pod \"limitador-limitador-8588749977-fsbjh\" (UID: \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\") " pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:45.240904 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.240860    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq27k\" (UniqueName: \"kubernetes.io/projected/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-kube-api-access-sq27k\") pod \"limitador-limitador-8588749977-fsbjh\" (UID: \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\") " pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:45.351721 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.351628    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:45.625871 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.625836    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-fsbjh"]
Sep 08 20:33:45.627386 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:33:45.627364    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619a7bcb_a756_44d9_bfcf_e05fc0c9a759.slice/crio-38e0e4d70996af9f1efede195b12b75090616df2e06389dfa7d39a6b92c7d343 WatchSource:0}: Error finding container 38e0e4d70996af9f1efede195b12b75090616df2e06389dfa7d39a6b92c7d343: Status 404 returned error can't find the container with id 38e0e4d70996af9f1efede195b12b75090616df2e06389dfa7d39a6b92c7d343
Sep 08 20:33:45.999335 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:45.999284    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh" event={"ID":"619a7bcb-a756-44d9-bfcf-e05fc0c9a759","Type":"ContainerStarted","Data":"38e0e4d70996af9f1efede195b12b75090616df2e06389dfa7d39a6b92c7d343"}
Sep 08 20:33:46.013841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.013803    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-px6w7"]
Sep 08 20:33:46.018592 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.018572    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-px6w7"
Sep 08 20:33:46.021596 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.021570    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l78c9\""
Sep 08 20:33:46.023182 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.023159    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-px6w7"]
Sep 08 20:33:46.138448 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.138410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txh5s\" (UniqueName: \"kubernetes.io/projected/487c8391-083f-4794-8b5a-b626fc57a163-kube-api-access-txh5s\") pod \"authorino-59c974bd48-px6w7\" (UID: \"487c8391-083f-4794-8b5a-b626fc57a163\") " pod="kuadrant-system/authorino-59c974bd48-px6w7"
Sep 08 20:33:46.239944 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.239875    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txh5s\" (UniqueName: \"kubernetes.io/projected/487c8391-083f-4794-8b5a-b626fc57a163-kube-api-access-txh5s\") pod \"authorino-59c974bd48-px6w7\" (UID: \"487c8391-083f-4794-8b5a-b626fc57a163\") " pod="kuadrant-system/authorino-59c974bd48-px6w7"
Sep 08 20:33:46.248144 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.248115    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txh5s\" (UniqueName: \"kubernetes.io/projected/487c8391-083f-4794-8b5a-b626fc57a163-kube-api-access-txh5s\") pod \"authorino-59c974bd48-px6w7\" (UID: \"487c8391-083f-4794-8b5a-b626fc57a163\") " pod="kuadrant-system/authorino-59c974bd48-px6w7"
Sep 08 20:33:46.329720 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.329629    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-px6w7"
Sep 08 20:33:46.599236 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:46.599209    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-px6w7"]
Sep 08 20:33:46.600818 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:33:46.600785    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod487c8391_083f_4794_8b5a_b626fc57a163.slice/crio-3722edbfb3ffe99e3f1733e57cdfb40e36b92477b457965ecd7d661f3ea4a9a6 WatchSource:0}: Error finding container 3722edbfb3ffe99e3f1733e57cdfb40e36b92477b457965ecd7d661f3ea4a9a6: Status 404 returned error can't find the container with id 3722edbfb3ffe99e3f1733e57cdfb40e36b92477b457965ecd7d661f3ea4a9a6
Sep 08 20:33:47.009457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:47.009411    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-px6w7" event={"ID":"487c8391-083f-4794-8b5a-b626fc57a163","Type":"ContainerStarted","Data":"3722edbfb3ffe99e3f1733e57cdfb40e36b92477b457965ecd7d661f3ea4a9a6"}
Sep 08 20:33:48.016678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:48.016642    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh" event={"ID":"619a7bcb-a756-44d9-bfcf-e05fc0c9a759","Type":"ContainerStarted","Data":"2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b"}
Sep 08 20:33:48.017196 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:48.016773    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:33:48.035348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:48.035290    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh" podStartSLOduration=1.629392159 podStartE2EDuration="3.035272622s" totalImagesPullingTime="1.405880463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:48.034736479 +0000 UTC m=+514.199907055" watchObservedRunningTime="2026-09-08 20:33:48.035272622 +0000 UTC m=+514.200443197"
Sep 08 20:33:59.024447 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:33:59.024409    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:34:00.194589 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.194543    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-fsbjh"]
Sep 08 20:34:00.195094 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.194812    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh" podUID="619a7bcb-a756-44d9-bfcf-e05fc0c9a759" containerName="limitador" containerID="cri-o://2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b" gracePeriod=30
Sep 08 20:34:00.751417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.751394    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:34:00.847575 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.847486    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq27k\" (UniqueName: \"kubernetes.io/projected/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-kube-api-access-sq27k\") pod \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\" (UID: \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\") "
Sep 08 20:34:00.847575 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.847542    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-config-file\") pod \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\" (UID: \"619a7bcb-a756-44d9-bfcf-e05fc0c9a759\") "
Sep 08 20:34:00.847966 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.847939    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-config-file" (OuterVolumeSpecName: "config-file") pod "619a7bcb-a756-44d9-bfcf-e05fc0c9a759" (UID: "619a7bcb-a756-44d9-bfcf-e05fc0c9a759"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:34:00.849693 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.849668    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-kube-api-access-sq27k" (OuterVolumeSpecName: "kube-api-access-sq27k") pod "619a7bcb-a756-44d9-bfcf-e05fc0c9a759" (UID: "619a7bcb-a756-44d9-bfcf-e05fc0c9a759"). InnerVolumeSpecName "kube-api-access-sq27k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:34:00.948257 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.948207    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq27k\" (UniqueName: \"kubernetes.io/projected/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-kube-api-access-sq27k\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:34:00.948257 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:00.948249    2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/619a7bcb-a756-44d9-bfcf-e05fc0c9a759-config-file\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:34:01.092585 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.092551    2561 generic.go:356] "Generic (PLEG): container finished" podID="619a7bcb-a756-44d9-bfcf-e05fc0c9a759" containerID="2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b" exitCode=0
Sep 08 20:34:01.092755 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.092600    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh" event={"ID":"619a7bcb-a756-44d9-bfcf-e05fc0c9a759","Type":"ContainerDied","Data":"2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b"}
Sep 08 20:34:01.092755 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.092638    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh" event={"ID":"619a7bcb-a756-44d9-bfcf-e05fc0c9a759","Type":"ContainerDied","Data":"38e0e4d70996af9f1efede195b12b75090616df2e06389dfa7d39a6b92c7d343"}
Sep 08 20:34:01.092755 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.092659    2561 scope.go:117] "RemoveContainer" containerID="2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b"
Sep 08 20:34:01.092755 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.092601    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-fsbjh"
Sep 08 20:34:01.102073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.102055    2561 scope.go:117] "RemoveContainer" containerID="2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b"
Sep 08 20:34:01.102364 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:34:01.102347    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b\": container with ID starting with 2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b not found: ID does not exist" containerID="2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b"
Sep 08 20:34:01.102434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.102373    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b"} err="failed to get container status \"2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b\": rpc error: code = NotFound desc = could not find container \"2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b\": container with ID starting with 2cb7d850dcf1114cdca00c693591bdfba169018c4a10badd5878e8ebb329c21b not found: ID does not exist"
Sep 08 20:34:01.118429 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.118393    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-fsbjh"]
Sep 08 20:34:01.124022 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:01.123995    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-fsbjh"]
Sep 08 20:34:02.365916 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:02.365867    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="619a7bcb-a756-44d9-bfcf-e05fc0c9a759" path="/var/lib/kubelet/pods/619a7bcb-a756-44d9-bfcf-e05fc0c9a759/volumes"
Sep 08 20:34:03.107602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:03.107560    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-px6w7" event={"ID":"487c8391-083f-4794-8b5a-b626fc57a163","Type":"ContainerStarted","Data":"3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60"}
Sep 08 20:34:03.124382 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:03.124320    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-px6w7" podStartSLOduration=2.280426615 podStartE2EDuration="18.124301211s" totalImagesPullingTime="15.843874596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:34:03.121644813 +0000 UTC m=+529.286815389" watchObservedRunningTime="2026-09-08 20:34:03.124301211 +0000 UTC m=+529.289471788"
Sep 08 20:34:06.668826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.668742    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"]
Sep 08 20:34:06.669397 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.669379    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619a7bcb-a756-44d9-bfcf-e05fc0c9a759" containerName="limitador"
Sep 08 20:34:06.669453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.669401    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="619a7bcb-a756-44d9-bfcf-e05fc0c9a759" containerName="limitador"
Sep 08 20:34:06.669539 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.669527    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="619a7bcb-a756-44d9-bfcf-e05fc0c9a759" containerName="limitador"
Sep 08 20:34:06.673569 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.673543    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:06.676151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.676126    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 08 20:34:06.676291 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.676157    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-gqrp9\""
Sep 08 20:34:06.676291 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.676194    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 20:34:06.677119 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.677104    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 20:34:06.682107 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.682085    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"]
Sep 08 20:34:06.701005 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.700975    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw4xr\" (UniqueName: \"kubernetes.io/projected/2ab28082-7dda-4daa-943c-7679b5d8b950-kube-api-access-mw4xr\") pod \"postgres-69fb4594cf-p59dk\" (UID: \"2ab28082-7dda-4daa-943c-7679b5d8b950\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:06.701160 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.701041    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ab28082-7dda-4daa-943c-7679b5d8b950-data\") pod \"postgres-69fb4594cf-p59dk\" (UID: \"2ab28082-7dda-4daa-943c-7679b5d8b950\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:06.802324 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.802283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw4xr\" (UniqueName: \"kubernetes.io/projected/2ab28082-7dda-4daa-943c-7679b5d8b950-kube-api-access-mw4xr\") pod \"postgres-69fb4594cf-p59dk\" (UID: \"2ab28082-7dda-4daa-943c-7679b5d8b950\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:06.802501 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.802357    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ab28082-7dda-4daa-943c-7679b5d8b950-data\") pod \"postgres-69fb4594cf-p59dk\" (UID: \"2ab28082-7dda-4daa-943c-7679b5d8b950\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:06.802692 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.802678    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ab28082-7dda-4daa-943c-7679b5d8b950-data\") pod \"postgres-69fb4594cf-p59dk\" (UID: \"2ab28082-7dda-4daa-943c-7679b5d8b950\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:06.810553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.810526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw4xr\" (UniqueName: \"kubernetes.io/projected/2ab28082-7dda-4daa-943c-7679b5d8b950-kube-api-access-mw4xr\") pod \"postgres-69fb4594cf-p59dk\" (UID: \"2ab28082-7dda-4daa-943c-7679b5d8b950\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:06.985941 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:06.985820    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:07.259835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:07.259803    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"]
Sep 08 20:34:07.262400 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:34:07.262357    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab28082_7dda_4daa_943c_7679b5d8b950.slice/crio-103cc463a8b3c2800eaa41cf9fc72fedd7eb112554d12518b128a5c3f91e8050 WatchSource:0}: Error finding container 103cc463a8b3c2800eaa41cf9fc72fedd7eb112554d12518b128a5c3f91e8050: Status 404 returned error can't find the container with id 103cc463a8b3c2800eaa41cf9fc72fedd7eb112554d12518b128a5c3f91e8050
Sep 08 20:34:08.137862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:08.137828    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk" event={"ID":"2ab28082-7dda-4daa-943c-7679b5d8b950","Type":"ContainerStarted","Data":"103cc463a8b3c2800eaa41cf9fc72fedd7eb112554d12518b128a5c3f91e8050"}
Sep 08 20:34:10.154236 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:10.154194    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk" event={"ID":"2ab28082-7dda-4daa-943c-7679b5d8b950","Type":"ContainerStarted","Data":"58feb36dc2a92e641809f43eed8d983ab089b61e24c484084b7097b1cea82909"}
Sep 08 20:34:10.154635 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:10.154351    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:10.172230 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:10.172163    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk" podStartSLOduration=1.7109990910000001 podStartE2EDuration="4.172139389s" totalImagesPullingTime="2.461140298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:34:10.170483728 +0000 UTC m=+536.335654304" watchObservedRunningTime="2026-09-08 20:34:10.172139389 +0000 UTC m=+536.337309965"
Sep 08 20:34:16.189594 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.189561    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p59dk"
Sep 08 20:34:16.816316 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.816280    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fdfkp"]
Sep 08 20:34:16.825075 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.825044    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:16.826482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.826454    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fdfkp"]
Sep 08 20:34:16.827601 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.827538    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 08 20:34:16.896235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.896196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxzm6\" (UniqueName: \"kubernetes.io/projected/2de15dc4-2bdf-489d-833d-ae30841f2e17-kube-api-access-qxzm6\") pod \"authorino-6d4dfc649f-fdfkp\" (UID: \"2de15dc4-2bdf-489d-833d-ae30841f2e17\") " pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:16.896407 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.896284    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2de15dc4-2bdf-489d-833d-ae30841f2e17-tls-cert\") pod \"authorino-6d4dfc649f-fdfkp\" (UID: \"2de15dc4-2bdf-489d-833d-ae30841f2e17\") " pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:16.945680 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.945640    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fdfkp"]
Sep 08 20:34:16.945989 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:34:16.945965    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qxzm6 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-fdfkp" podUID="2de15dc4-2bdf-489d-833d-ae30841f2e17"
Sep 08 20:34:16.971696 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.971657    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-796f9c8c55-8cdzw"]
Sep 08 20:34:16.975675 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.975649    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:34:16.983508 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:16.983476    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-796f9c8c55-8cdzw"]
Sep 08 20:34:17.001194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.000689    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2de15dc4-2bdf-489d-833d-ae30841f2e17-tls-cert\") pod \"authorino-6d4dfc649f-fdfkp\" (UID: \"2de15dc4-2bdf-489d-833d-ae30841f2e17\") " pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:17.001194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.000816    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzm6\" (UniqueName: \"kubernetes.io/projected/2de15dc4-2bdf-489d-833d-ae30841f2e17-kube-api-access-qxzm6\") pod \"authorino-6d4dfc649f-fdfkp\" (UID: \"2de15dc4-2bdf-489d-833d-ae30841f2e17\") " pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:17.004170 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.004144    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2de15dc4-2bdf-489d-833d-ae30841f2e17-tls-cert\") pod \"authorino-6d4dfc649f-fdfkp\" (UID: \"2de15dc4-2bdf-489d-833d-ae30841f2e17\") " pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:17.008804 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.008777    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxzm6\" (UniqueName: \"kubernetes.io/projected/2de15dc4-2bdf-489d-833d-ae30841f2e17-kube-api-access-qxzm6\") pod \"authorino-6d4dfc649f-fdfkp\" (UID: \"2de15dc4-2bdf-489d-833d-ae30841f2e17\") " pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:17.101597 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.101510    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91368725-17db-4b56-b1bf-361151132ca7-tls-cert\") pod \"authorino-796f9c8c55-8cdzw\" (UID: \"91368725-17db-4b56-b1bf-361151132ca7\") " pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:34:17.101597 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.101572    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqd7b\" (UniqueName: \"kubernetes.io/projected/91368725-17db-4b56-b1bf-361151132ca7-kube-api-access-cqd7b\") pod \"authorino-796f9c8c55-8cdzw\" (UID: \"91368725-17db-4b56-b1bf-361151132ca7\") " pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:34:17.194463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.194426    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:17.202357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.202328    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:17.202766 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.202746    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqd7b\" (UniqueName: \"kubernetes.io/projected/91368725-17db-4b56-b1bf-361151132ca7-kube-api-access-cqd7b\") pod \"authorino-796f9c8c55-8cdzw\" (UID: \"91368725-17db-4b56-b1bf-361151132ca7\") " pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:34:17.202886 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.202870    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91368725-17db-4b56-b1bf-361151132ca7-tls-cert\") pod \"authorino-796f9c8c55-8cdzw\" (UID: \"91368725-17db-4b56-b1bf-361151132ca7\") " pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:34:17.205247 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.205231    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91368725-17db-4b56-b1bf-361151132ca7-tls-cert\") pod \"authorino-796f9c8c55-8cdzw\" (UID: \"91368725-17db-4b56-b1bf-361151132ca7\") " pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:34:17.213786 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.213766    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqd7b\" (UniqueName: \"kubernetes.io/projected/91368725-17db-4b56-b1bf-361151132ca7-kube-api-access-cqd7b\") pod \"authorino-796f9c8c55-8cdzw\" (UID: \"91368725-17db-4b56-b1bf-361151132ca7\") " pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:34:17.286795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.286754    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:34:17.303525 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.303497    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2de15dc4-2bdf-489d-833d-ae30841f2e17-tls-cert\") pod \"2de15dc4-2bdf-489d-833d-ae30841f2e17\" (UID: \"2de15dc4-2bdf-489d-833d-ae30841f2e17\") "
Sep 08 20:34:17.303649 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.303547    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxzm6\" (UniqueName: \"kubernetes.io/projected/2de15dc4-2bdf-489d-833d-ae30841f2e17-kube-api-access-qxzm6\") pod \"2de15dc4-2bdf-489d-833d-ae30841f2e17\" (UID: \"2de15dc4-2bdf-489d-833d-ae30841f2e17\") "
Sep 08 20:34:17.305811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.305786    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de15dc4-2bdf-489d-833d-ae30841f2e17-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2de15dc4-2bdf-489d-833d-ae30841f2e17" (UID: "2de15dc4-2bdf-489d-833d-ae30841f2e17"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:34:17.305963 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.305817    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de15dc4-2bdf-489d-833d-ae30841f2e17-kube-api-access-qxzm6" (OuterVolumeSpecName: "kube-api-access-qxzm6") pod "2de15dc4-2bdf-489d-833d-ae30841f2e17" (UID: "2de15dc4-2bdf-489d-833d-ae30841f2e17"). InnerVolumeSpecName "kube-api-access-qxzm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:34:17.404776 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.404741    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2de15dc4-2bdf-489d-833d-ae30841f2e17-tls-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:34:17.404964 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.404780    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxzm6\" (UniqueName: \"kubernetes.io/projected/2de15dc4-2bdf-489d-833d-ae30841f2e17-kube-api-access-qxzm6\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:34:17.557728 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:17.557698    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-796f9c8c55-8cdzw"]
Sep 08 20:34:17.559191 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:34:17.559161    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91368725_17db_4b56_b1bf_361151132ca7.slice/crio-96eb7f41fb2629920e8c71b4436f0705f037564b0dea6b4ba816b1a9e823e611 WatchSource:0}: Error finding container 96eb7f41fb2629920e8c71b4436f0705f037564b0dea6b4ba816b1a9e823e611: Status 404 returned error can't find the container with id 96eb7f41fb2629920e8c71b4436f0705f037564b0dea6b4ba816b1a9e823e611
Sep 08 20:34:18.202668 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.202622    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-796f9c8c55-8cdzw" event={"ID":"91368725-17db-4b56-b1bf-361151132ca7","Type":"ContainerStarted","Data":"02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b"}
Sep 08 20:34:18.203109 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.202685    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-796f9c8c55-8cdzw" event={"ID":"91368725-17db-4b56-b1bf-361151132ca7","Type":"ContainerStarted","Data":"96eb7f41fb2629920e8c71b4436f0705f037564b0dea6b4ba816b1a9e823e611"}
Sep 08 20:34:18.203109 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.202688    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-fdfkp"
Sep 08 20:34:18.219227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.219177    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-796f9c8c55-8cdzw" podStartSLOduration=1.726329052 podStartE2EDuration="2.219161481s" totalImagesPullingTime="492.832429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:34:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:34:18.217875601 +0000 UTC m=+544.383046177" watchObservedRunningTime="2026-09-08 20:34:18.219161481 +0000 UTC m=+544.384332057"
Sep 08 20:34:18.250239 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.250201    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fdfkp"]
Sep 08 20:34:18.253831 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.253798    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-px6w7"]
Sep 08 20:34:18.254045 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.254009    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-px6w7" podUID="487c8391-083f-4794-8b5a-b626fc57a163" containerName="authorino" containerID="cri-o://3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60" gracePeriod=30
Sep 08 20:34:18.256668 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.256645    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fdfkp"]
Sep 08 20:34:18.365657 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.365620    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de15dc4-2bdf-489d-833d-ae30841f2e17" path="/var/lib/kubelet/pods/2de15dc4-2bdf-489d-833d-ae30841f2e17/volumes"
Sep 08 20:34:18.502285 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.502260    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-px6w7"
Sep 08 20:34:18.616538 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.616450    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txh5s\" (UniqueName: \"kubernetes.io/projected/487c8391-083f-4794-8b5a-b626fc57a163-kube-api-access-txh5s\") pod \"487c8391-083f-4794-8b5a-b626fc57a163\" (UID: \"487c8391-083f-4794-8b5a-b626fc57a163\") "
Sep 08 20:34:18.618553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.618519    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/487c8391-083f-4794-8b5a-b626fc57a163-kube-api-access-txh5s" (OuterVolumeSpecName: "kube-api-access-txh5s") pod "487c8391-083f-4794-8b5a-b626fc57a163" (UID: "487c8391-083f-4794-8b5a-b626fc57a163"). InnerVolumeSpecName "kube-api-access-txh5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:34:18.717304 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:18.717271    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txh5s\" (UniqueName: \"kubernetes.io/projected/487c8391-083f-4794-8b5a-b626fc57a163-kube-api-access-txh5s\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:34:19.210761 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.210726    2561 generic.go:356] "Generic (PLEG): container finished" podID="487c8391-083f-4794-8b5a-b626fc57a163" containerID="3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60" exitCode=0
Sep 08 20:34:19.211295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.210768    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-px6w7"
Sep 08 20:34:19.211295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.210807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-px6w7" event={"ID":"487c8391-083f-4794-8b5a-b626fc57a163","Type":"ContainerDied","Data":"3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60"}
Sep 08 20:34:19.211295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.210844    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-px6w7" event={"ID":"487c8391-083f-4794-8b5a-b626fc57a163","Type":"ContainerDied","Data":"3722edbfb3ffe99e3f1733e57cdfb40e36b92477b457965ecd7d661f3ea4a9a6"}
Sep 08 20:34:19.211295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.210866    2561 scope.go:117] "RemoveContainer" containerID="3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60"
Sep 08 20:34:19.220228 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.220207    2561 scope.go:117] "RemoveContainer" containerID="3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60"
Sep 08 20:34:19.220544 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:34:19.220524    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60\": container with ID starting with 3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60 not found: ID does not exist" containerID="3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60"
Sep 08 20:34:19.220616 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.220560    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60"} err="failed to get container status \"3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60\": rpc error: code = NotFound desc = could not find container \"3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60\": container with ID starting with 3fb636243bac205e4470694de9b10e575d05b872db14970e888e344274b1bf60 not found: ID does not exist"
Sep 08 20:34:19.232989 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.232957    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-px6w7"]
Sep 08 20:34:19.237139 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:19.237111    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-px6w7"]
Sep 08 20:34:20.363520 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:20.363486    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="487c8391-083f-4794-8b5a-b626fc57a163" path="/var/lib/kubelet/pods/487c8391-083f-4794-8b5a-b626fc57a163/volumes"
Sep 08 20:34:23.020599 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.020548    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59b7dd6d9b-k9p4w"]
Sep 08 20:34:23.021221 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.021170    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487c8391-083f-4794-8b5a-b626fc57a163" containerName="authorino"
Sep 08 20:34:23.021221 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.021198    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="487c8391-083f-4794-8b5a-b626fc57a163" containerName="authorino"
Sep 08 20:34:23.021439 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.021325    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="487c8391-083f-4794-8b5a-b626fc57a163" containerName="authorino"
Sep 08 20:34:23.026644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.026625    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.029341 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.029319    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 20:34:23.029341 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.029326    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 20:34:23.029511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.029326    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 20:34:23.029511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.029498    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qwqtd\""
Sep 08 20:34:23.035000 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.034979    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b7dd6d9b-k9p4w"]
Sep 08 20:34:23.160294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.160244    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-webhook-cert\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.160483 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.160299    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g94ms\" (UniqueName: \"kubernetes.io/projected/203aebd6-5ec0-47e7-b545-b9c240369d19-kube-api-access-g94ms\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.160483 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.160432    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.200157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.200121    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-dxwnd"]
Sep 08 20:34:23.204170 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.204146    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.215406 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.215374    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-dxwnd"]
Sep 08 20:34:23.261098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.261069    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-webhook-cert\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.261098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.261101    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g94ms\" (UniqueName: \"kubernetes.io/projected/203aebd6-5ec0-47e7-b545-b9c240369d19-kube-api-access-g94ms\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.261352 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.261124    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-metrics-certs\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.261352 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.261248    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvg5\" (UniqueName: \"kubernetes.io/projected/d004915c-96ee-4263-bd9f-65258d2b2cc1-kube-api-access-7cvg5\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.261457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.261394    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.261519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.261460    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-webhook-cert\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.261519 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:34:23.261510    2561 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 08 20:34:23.261595 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:34:23.261568    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs podName:203aebd6-5ec0-47e7-b545-b9c240369d19 nodeName:}" failed. No retries permitted until 2026-09-08 20:34:23.761548995 +0000 UTC m=+549.926719553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs") pod "maas-controller-59b7dd6d9b-k9p4w" (UID: "203aebd6-5ec0-47e7-b545-b9c240369d19") : secret "maas-controller-metrics-tls" not found
Sep 08 20:34:23.263639 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.263620    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-webhook-cert\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.272274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.272219    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g94ms\" (UniqueName: \"kubernetes.io/projected/203aebd6-5ec0-47e7-b545-b9c240369d19-kube-api-access-g94ms\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.362730 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.362698    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-webhook-cert\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.362890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.362750    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-metrics-certs\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.362987 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.362889    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvg5\" (UniqueName: \"kubernetes.io/projected/d004915c-96ee-4263-bd9f-65258d2b2cc1-kube-api-access-7cvg5\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.365323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.365295    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-webhook-cert\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.365449 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.365325    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-metrics-certs\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.372848 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.372824    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvg5\" (UniqueName: \"kubernetes.io/projected/d004915c-96ee-4263-bd9f-65258d2b2cc1-kube-api-access-7cvg5\") pod \"maas-controller-7cc45c6c68-dxwnd\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") " pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.516025 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.515988    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:23.766913 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.766852    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.769836 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.769810    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs\") pod \"maas-controller-59b7dd6d9b-k9p4w\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") " pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:23.788598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.788571    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-dxwnd"]
Sep 08 20:34:23.790243 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:34:23.790209    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd004915c_96ee_4263_bd9f_65258d2b2cc1.slice/crio-497ac8135f7ac86048c1ca23f075384420486b0dad0c41b43806bc5c144ad246 WatchSource:0}: Error finding container 497ac8135f7ac86048c1ca23f075384420486b0dad0c41b43806bc5c144ad246: Status 404 returned error can't find the container with id 497ac8135f7ac86048c1ca23f075384420486b0dad0c41b43806bc5c144ad246
Sep 08 20:34:23.939630 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:23.939594    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:24.212966 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:24.212938    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b7dd6d9b-k9p4w"]
Sep 08 20:34:24.213940 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:34:24.213915    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203aebd6_5ec0_47e7_b545_b9c240369d19.slice/crio-3317cbb175a9326db672afc4c40c6a6b4a778ef994960d1ac8d8a2a7c83716ab WatchSource:0}: Error finding container 3317cbb175a9326db672afc4c40c6a6b4a778ef994960d1ac8d8a2a7c83716ab: Status 404 returned error can't find the container with id 3317cbb175a9326db672afc4c40c6a6b4a778ef994960d1ac8d8a2a7c83716ab
Sep 08 20:34:24.241768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:24.241730    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd" event={"ID":"d004915c-96ee-4263-bd9f-65258d2b2cc1","Type":"ContainerStarted","Data":"497ac8135f7ac86048c1ca23f075384420486b0dad0c41b43806bc5c144ad246"}
Sep 08 20:34:24.242939 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:24.242915    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w" event={"ID":"203aebd6-5ec0-47e7-b545-b9c240369d19","Type":"ContainerStarted","Data":"3317cbb175a9326db672afc4c40c6a6b4a778ef994960d1ac8d8a2a7c83716ab"}
Sep 08 20:34:26.262302 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:26.262261    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd" event={"ID":"d004915c-96ee-4263-bd9f-65258d2b2cc1","Type":"ContainerStarted","Data":"835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f"}
Sep 08 20:34:26.262803 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:26.262338    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:26.264125 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:26.264103    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w" event={"ID":"203aebd6-5ec0-47e7-b545-b9c240369d19","Type":"ContainerStarted","Data":"bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036"}
Sep 08 20:34:26.264258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:26.264228    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:26.300879 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:26.300811    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd" podStartSLOduration=1.22503434 podStartE2EDuration="3.3007879s" totalImagesPullingTime="2.07575356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:34:26.297858353 +0000 UTC m=+552.463028931" watchObservedRunningTime="2026-09-08 20:34:26.3007879 +0000 UTC m=+552.465958478"
Sep 08 20:34:26.320979 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:26.320924    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w" podStartSLOduration=2.665716057 podStartE2EDuration="4.320904214s" totalImagesPullingTime="1.655188157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:34:26.31905949 +0000 UTC m=+552.484230069" watchObservedRunningTime="2026-09-08 20:34:26.320904214 +0000 UTC m=+552.486074784"
Sep 08 20:34:37.271474 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.271440    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:34:37.272018 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.271524    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:37.338601 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.338567    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59b7dd6d9b-k9p4w"]
Sep 08 20:34:37.338813 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.338786    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w" podUID="203aebd6-5ec0-47e7-b545-b9c240369d19" containerName="manager" containerID="cri-o://bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036" gracePeriod=30
Sep 08 20:34:37.590378 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.590353    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:37.698272 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.698238    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g94ms\" (UniqueName: \"kubernetes.io/projected/203aebd6-5ec0-47e7-b545-b9c240369d19-kube-api-access-g94ms\") pod \"203aebd6-5ec0-47e7-b545-b9c240369d19\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") "
Sep 08 20:34:37.698272 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.698275    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-webhook-cert\") pod \"203aebd6-5ec0-47e7-b545-b9c240369d19\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") "
Sep 08 20:34:37.698540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.698326    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs\") pod \"203aebd6-5ec0-47e7-b545-b9c240369d19\" (UID: \"203aebd6-5ec0-47e7-b545-b9c240369d19\") "
Sep 08 20:34:37.700579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.700539    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "203aebd6-5ec0-47e7-b545-b9c240369d19" (UID: "203aebd6-5ec0-47e7-b545-b9c240369d19"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:34:37.700688 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.700617    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/203aebd6-5ec0-47e7-b545-b9c240369d19-kube-api-access-g94ms" (OuterVolumeSpecName: "kube-api-access-g94ms") pod "203aebd6-5ec0-47e7-b545-b9c240369d19" (UID: "203aebd6-5ec0-47e7-b545-b9c240369d19"). InnerVolumeSpecName "kube-api-access-g94ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:34:37.700688 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.700618    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "203aebd6-5ec0-47e7-b545-b9c240369d19" (UID: "203aebd6-5ec0-47e7-b545-b9c240369d19"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:34:37.799151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.799112    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g94ms\" (UniqueName: \"kubernetes.io/projected/203aebd6-5ec0-47e7-b545-b9c240369d19-kube-api-access-g94ms\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:34:37.799151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.799141    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-webhook-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:34:37.799151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:37.799151    2561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203aebd6-5ec0-47e7-b545-b9c240369d19-metrics-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:34:38.336165 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.336127    2561 generic.go:356] "Generic (PLEG): container finished" podID="203aebd6-5ec0-47e7-b545-b9c240369d19" containerID="bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036" exitCode=0
Sep 08 20:34:38.336548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.336181    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w"
Sep 08 20:34:38.336548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.336186    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w" event={"ID":"203aebd6-5ec0-47e7-b545-b9c240369d19","Type":"ContainerDied","Data":"bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036"}
Sep 08 20:34:38.336548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.336290    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b7dd6d9b-k9p4w" event={"ID":"203aebd6-5ec0-47e7-b545-b9c240369d19","Type":"ContainerDied","Data":"3317cbb175a9326db672afc4c40c6a6b4a778ef994960d1ac8d8a2a7c83716ab"}
Sep 08 20:34:38.336548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.336318    2561 scope.go:117] "RemoveContainer" containerID="bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036"
Sep 08 20:34:38.346223 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.346058    2561 scope.go:117] "RemoveContainer" containerID="bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036"
Sep 08 20:34:38.346369 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:34:38.346349    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036\": container with ID starting with bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036 not found: ID does not exist" containerID="bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036"
Sep 08 20:34:38.346416 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.346377    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036"} err="failed to get container status \"bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036\": rpc error: code = NotFound desc = could not find container \"bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036\": container with ID starting with bdaf1636c5ec41fdebcfa2a3db0749c6c7211986af87337cd49a6619f080a036 not found: ID does not exist"
Sep 08 20:34:38.363910 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.363870    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59b7dd6d9b-k9p4w"]
Sep 08 20:34:38.364047 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:38.363937    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59b7dd6d9b-k9p4w"]
Sep 08 20:34:40.363852 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:34:40.363817    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="203aebd6-5ec0-47e7-b545-b9c240369d19" path="/var/lib/kubelet/pods/203aebd6-5ec0-47e7-b545-b9c240369d19/volumes"
Sep 08 20:35:00.646643 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.646603    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"]
Sep 08 20:35:00.647833 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.647806    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="203aebd6-5ec0-47e7-b545-b9c240369d19" containerName="manager"
Sep 08 20:35:00.647833 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.647839    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="203aebd6-5ec0-47e7-b545-b9c240369d19" containerName="manager"
Sep 08 20:35:00.648044 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.648037    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="203aebd6-5ec0-47e7-b545-b9c240369d19" containerName="manager"
Sep 08 20:35:00.659393 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.659334    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:00.661445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.661417    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"]
Sep 08 20:35:00.664460 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.664437    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m9947\""
Sep 08 20:35:00.664766 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.664746    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 08 20:35:00.677878 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.677852    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"]
Sep 08 20:35:00.682157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.682136    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:00.690001 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.689805    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"]
Sep 08 20:35:00.710586 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.710549    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkx92\" (UniqueName: \"kubernetes.io/projected/b2e50305-6c7a-4b87-8f30-fe9e06b6c367-kube-api-access-dkx92\") pod \"payload-pre-processing-59fbc47bf8-rv2lp\" (UID: \"b2e50305-6c7a-4b87-8f30-fe9e06b6c367\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:00.710759 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.710637    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e50305-6c7a-4b87-8f30-fe9e06b6c367-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rv2lp\" (UID: \"b2e50305-6c7a-4b87-8f30-fe9e06b6c367\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:00.812077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.812039    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e50305-6c7a-4b87-8f30-fe9e06b6c367-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rv2lp\" (UID: \"b2e50305-6c7a-4b87-8f30-fe9e06b6c367\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:00.812275 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.812153    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b6128d-c751-4948-8312-f0ed78e55355-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-mw9v6\" (UID: \"e7b6128d-c751-4948-8312-f0ed78e55355\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:00.812275 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.812196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjxxx\" (UniqueName: \"kubernetes.io/projected/e7b6128d-c751-4948-8312-f0ed78e55355-kube-api-access-xjxxx\") pod \"payload-processing-5b54b7fbb7-mw9v6\" (UID: \"e7b6128d-c751-4948-8312-f0ed78e55355\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:00.812401 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.812342    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkx92\" (UniqueName: \"kubernetes.io/projected/b2e50305-6c7a-4b87-8f30-fe9e06b6c367-kube-api-access-dkx92\") pod \"payload-pre-processing-59fbc47bf8-rv2lp\" (UID: \"b2e50305-6c7a-4b87-8f30-fe9e06b6c367\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:00.812976 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.812945    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e50305-6c7a-4b87-8f30-fe9e06b6c367-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rv2lp\" (UID: \"b2e50305-6c7a-4b87-8f30-fe9e06b6c367\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:00.820866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.820834    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkx92\" (UniqueName: \"kubernetes.io/projected/b2e50305-6c7a-4b87-8f30-fe9e06b6c367-kube-api-access-dkx92\") pod \"payload-pre-processing-59fbc47bf8-rv2lp\" (UID: \"b2e50305-6c7a-4b87-8f30-fe9e06b6c367\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:00.913342 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.913299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b6128d-c751-4948-8312-f0ed78e55355-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-mw9v6\" (UID: \"e7b6128d-c751-4948-8312-f0ed78e55355\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:00.913560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.913361    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjxxx\" (UniqueName: \"kubernetes.io/projected/e7b6128d-c751-4948-8312-f0ed78e55355-kube-api-access-xjxxx\") pod \"payload-processing-5b54b7fbb7-mw9v6\" (UID: \"e7b6128d-c751-4948-8312-f0ed78e55355\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:00.914087 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.914065    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b6128d-c751-4948-8312-f0ed78e55355-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-mw9v6\" (UID: \"e7b6128d-c751-4948-8312-f0ed78e55355\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:00.924943 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.924886    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjxxx\" (UniqueName: \"kubernetes.io/projected/e7b6128d-c751-4948-8312-f0ed78e55355-kube-api-access-xjxxx\") pod \"payload-processing-5b54b7fbb7-mw9v6\" (UID: \"e7b6128d-c751-4948-8312-f0ed78e55355\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:00.971724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.971692    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:00.994460 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:00.994381    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:01.262535 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:01.262504    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"]
Sep 08 20:35:01.263349 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:35:01.263326    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e50305_6c7a_4b87_8f30_fe9e06b6c367.slice/crio-cf8097ab10f31d490f3e78312d996cfb0708c6d21f037a73c0c37fa2ab6bbbe3 WatchSource:0}: Error finding container cf8097ab10f31d490f3e78312d996cfb0708c6d21f037a73c0c37fa2ab6bbbe3: Status 404 returned error can't find the container with id cf8097ab10f31d490f3e78312d996cfb0708c6d21f037a73c0c37fa2ab6bbbe3
Sep 08 20:35:01.294077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:01.294044    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"]
Sep 08 20:35:01.295201 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:35:01.295176    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b6128d_c751_4948_8312_f0ed78e55355.slice/crio-c5fc88854e85d0569353fb88267bff26c8258b98d6cac068d1fee5fcd458cd44 WatchSource:0}: Error finding container c5fc88854e85d0569353fb88267bff26c8258b98d6cac068d1fee5fcd458cd44: Status 404 returned error can't find the container with id c5fc88854e85d0569353fb88267bff26c8258b98d6cac068d1fee5fcd458cd44
Sep 08 20:35:01.475761 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:01.475719    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp" event={"ID":"b2e50305-6c7a-4b87-8f30-fe9e06b6c367","Type":"ContainerStarted","Data":"cf8097ab10f31d490f3e78312d996cfb0708c6d21f037a73c0c37fa2ab6bbbe3"}
Sep 08 20:35:01.477025 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:01.477003    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6" event={"ID":"e7b6128d-c751-4948-8312-f0ed78e55355","Type":"ContainerStarted","Data":"c5fc88854e85d0569353fb88267bff26c8258b98d6cac068d1fee5fcd458cd44"}
Sep 08 20:35:04.505633 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:04.505588    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp" event={"ID":"b2e50305-6c7a-4b87-8f30-fe9e06b6c367","Type":"ContainerStarted","Data":"cb549a5c4e327ffc38d9bc066bee52250543017943d6390d243f7b467db177db"}
Sep 08 20:35:04.506140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:04.505723    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:04.507295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:04.507270    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6" event={"ID":"e7b6128d-c751-4948-8312-f0ed78e55355","Type":"ContainerStarted","Data":"99f73658a92eacae4573b3497f90a598972d062bbc5a62b92776e935ee27c8bf"}
Sep 08 20:35:04.507443 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:04.507357    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:04.522926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:04.522867    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp" podStartSLOduration=2.071918189 podStartE2EDuration="4.522850656s" totalImagesPullingTime="2.450932467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:35:04.521808585 +0000 UTC m=+590.686979162" watchObservedRunningTime="2026-09-08 20:35:04.522850656 +0000 UTC m=+590.688021232"
Sep 08 20:35:04.543916 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:04.540056    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6" podStartSLOduration=2.127285545 podStartE2EDuration="4.540032113s" totalImagesPullingTime="2.412746568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:35:04.538482294 +0000 UTC m=+590.703652872" watchObservedRunningTime="2026-09-08 20:35:04.540032113 +0000 UTC m=+590.705202690"
Sep 08 20:35:10.549048 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:10.549011    2561 generic.go:356] "Generic (PLEG): container finished" podID="ddd858c1-029d-4d85-ac36-dce67c4f4ebf" containerID="67ddcda28d151205047aea658311774fb8ab642ffaae5113fc9ddff2510659da" exitCode=0
Sep 08 20:35:10.549488 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:10.549087    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" event={"ID":"ddd858c1-029d-4d85-ac36-dce67c4f4ebf","Type":"ContainerDied","Data":"67ddcda28d151205047aea658311774fb8ab642ffaae5113fc9ddff2510659da"}
Sep 08 20:35:10.549541 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:10.549510    2561 scope.go:117] "RemoveContainer" containerID="67ddcda28d151205047aea658311774fb8ab642ffaae5113fc9ddff2510659da"
Sep 08 20:35:10.550541 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:10.550514    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:35:10.550614 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:10.550583    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:35:10.550652 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:10.550619    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:35:11.561698 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:11.561654    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" event={"ID":"ddd858c1-029d-4d85-ac36-dce67c4f4ebf","Type":"ContainerStarted","Data":"a88fb02417ffe35db209fc911e736f9d6072a042190d6f760100476b6eef476b"}
Sep 08 20:35:12.505548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:12.505505    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:35:12.506806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:12.506777    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 08 20:35:12.506945 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:12.506831    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" podUID="ddd858c1-029d-4d85-ac36-dce67c4f4ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 08 20:35:13.506076 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:13.506037    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 08 20:35:13.506475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:13.506100    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" podUID="ddd858c1-029d-4d85-ac36-dce67c4f4ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 08 20:35:14.314695 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:14.314663    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:35:14.314874 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:14.314804    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:35:14.505138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:14.505099    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 08 20:35:14.505326 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:14.505163    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" podUID="ddd858c1-029d-4d85-ac36-dce67c4f4ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 08 20:35:14.992361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:14.992307    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:35:15.505079 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:15.505016    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 08 20:35:15.505079 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:15.505068    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" podUID="ddd858c1-029d-4d85-ac36-dce67c4f4ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 08 20:35:15.516065 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:15.516031    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-mw9v6"
Sep 08 20:35:15.516393 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:15.516375    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rv2lp"
Sep 08 20:35:16.505209 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:16.505168    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 08 20:35:16.505696 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:16.505234    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj" podUID="ddd858c1-029d-4d85-ac36-dce67c4f4ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 08 20:35:17.509594 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:17.509564    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:35:17.510698 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:17.510679    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kv8wj"
Sep 08 20:35:39.591303 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.591186    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"]
Sep 08 20:35:39.595195 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.595170    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.598811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.598785    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 08 20:35:39.598811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.598806    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 20:35:39.599049 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.598962    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 20:35:39.599049 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.599026    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xsfxp\""
Sep 08 20:35:39.605153 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.605112    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"]
Sep 08 20:35:39.704653 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.704618    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4cc211-d382-4cf9-b137-3b204967776a-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.704847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.704680    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.704847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.704702    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.704847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.704741    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.704847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.704760    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56qbh\" (UniqueName: \"kubernetes.io/projected/4b4cc211-d382-4cf9-b137-3b204967776a-kube-api-access-56qbh\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.704847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.704810    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806239 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806199    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4cc211-d382-4cf9-b137-3b204967776a-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806285    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806314    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806401    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56qbh\" (UniqueName: \"kubernetes.io/projected/4b4cc211-d382-4cf9-b137-3b204967776a-kube-api-access-56qbh\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806783    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806879 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806814    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.806879 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.806847    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.808729 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.808706    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4cc211-d382-4cf9-b137-3b204967776a-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.809017 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.808999    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4cc211-d382-4cf9-b137-3b204967776a-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.814329 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.814303    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56qbh\" (UniqueName: \"kubernetes.io/projected/4b4cc211-d382-4cf9-b137-3b204967776a-kube-api-access-56qbh\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx\" (UID: \"4b4cc211-d382-4cf9-b137-3b204967776a\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:39.907444 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:39.907402    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:40.191802 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:40.191768    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"]
Sep 08 20:35:40.193186 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:35:40.193148    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4cc211_d382_4cf9_b137_3b204967776a.slice/crio-129980c6552007b292c117a8166f4bc2f0b9a1772f4676f1d9f5cf1a8d417562 WatchSource:0}: Error finding container 129980c6552007b292c117a8166f4bc2f0b9a1772f4676f1d9f5cf1a8d417562: Status 404 returned error can't find the container with id 129980c6552007b292c117a8166f4bc2f0b9a1772f4676f1d9f5cf1a8d417562
Sep 08 20:35:40.194790 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:40.194769    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:35:40.752418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:40.752378    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx" event={"ID":"4b4cc211-d382-4cf9-b137-3b204967776a","Type":"ContainerStarted","Data":"129980c6552007b292c117a8166f4bc2f0b9a1772f4676f1d9f5cf1a8d417562"}
Sep 08 20:35:41.759966 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:41.759926    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx" event={"ID":"4b4cc211-d382-4cf9-b137-3b204967776a","Type":"ContainerStarted","Data":"11a59514fffcca77f18ea69f7badc80da8a18861f832bfe1dd7560a3de974342"}
Sep 08 20:35:41.783070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:41.783007    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx" podStartSLOduration=1.7095239 podStartE2EDuration="2.782986259s" totalImagesPullingTime="1.073462359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:35:41.780663725 +0000 UTC m=+627.945834300" watchObservedRunningTime="2026-09-08 20:35:41.782986259 +0000 UTC m=+627.948156836"
Sep 08 20:35:49.908002 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:49.907959    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:49.908504 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:49.908028    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:49.928012 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:49.927980    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:35:50.831768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:35:50.831735    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx"
Sep 08 20:38:00.341808 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.341770    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-796f9c8c55-8cdzw"]
Sep 08 20:38:00.342413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.342000    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-796f9c8c55-8cdzw" podUID="91368725-17db-4b56-b1bf-361151132ca7" containerName="authorino" containerID="cri-o://02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b" gracePeriod=30
Sep 08 20:38:00.599258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.599189    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:38:00.606356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.606318    2561 generic.go:356] "Generic (PLEG): container finished" podID="91368725-17db-4b56-b1bf-361151132ca7" containerID="02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b" exitCode=0
Sep 08 20:38:00.606543 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.606399    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-796f9c8c55-8cdzw"
Sep 08 20:38:00.606543 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.606434    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-796f9c8c55-8cdzw" event={"ID":"91368725-17db-4b56-b1bf-361151132ca7","Type":"ContainerDied","Data":"02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b"}
Sep 08 20:38:00.606543 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.606483    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-796f9c8c55-8cdzw" event={"ID":"91368725-17db-4b56-b1bf-361151132ca7","Type":"ContainerDied","Data":"96eb7f41fb2629920e8c71b4436f0705f037564b0dea6b4ba816b1a9e823e611"}
Sep 08 20:38:00.606543 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.606508    2561 scope.go:117] "RemoveContainer" containerID="02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b"
Sep 08 20:38:00.618871 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.618844    2561 scope.go:117] "RemoveContainer" containerID="02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b"
Sep 08 20:38:00.619294 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:38:00.619274    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b\": container with ID starting with 02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b not found: ID does not exist" containerID="02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b"
Sep 08 20:38:00.619409 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.619303    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b"} err="failed to get container status \"02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b\": rpc error: code = NotFound desc = could not find container \"02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b\": container with ID starting with 02485a69a387d1d43816caf1edb338c995b8c9d3f2eb9601ce0a23c7c818699b not found: ID does not exist"
Sep 08 20:38:00.756791 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.756758    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqd7b\" (UniqueName: \"kubernetes.io/projected/91368725-17db-4b56-b1bf-361151132ca7-kube-api-access-cqd7b\") pod \"91368725-17db-4b56-b1bf-361151132ca7\" (UID: \"91368725-17db-4b56-b1bf-361151132ca7\") "
Sep 08 20:38:00.756986 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.756807    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91368725-17db-4b56-b1bf-361151132ca7-tls-cert\") pod \"91368725-17db-4b56-b1bf-361151132ca7\" (UID: \"91368725-17db-4b56-b1bf-361151132ca7\") "
Sep 08 20:38:00.759025 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.758988    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91368725-17db-4b56-b1bf-361151132ca7-kube-api-access-cqd7b" (OuterVolumeSpecName: "kube-api-access-cqd7b") pod "91368725-17db-4b56-b1bf-361151132ca7" (UID: "91368725-17db-4b56-b1bf-361151132ca7"). InnerVolumeSpecName "kube-api-access-cqd7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:38:00.768003 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.767967    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91368725-17db-4b56-b1bf-361151132ca7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "91368725-17db-4b56-b1bf-361151132ca7" (UID: "91368725-17db-4b56-b1bf-361151132ca7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:38:00.858458 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.858358    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91368725-17db-4b56-b1bf-361151132ca7-tls-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:38:00.858458 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.858394    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqd7b\" (UniqueName: \"kubernetes.io/projected/91368725-17db-4b56-b1bf-361151132ca7-kube-api-access-cqd7b\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:38:00.928317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.928277    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-796f9c8c55-8cdzw"]
Sep 08 20:38:00.935888 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:00.935859    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-796f9c8c55-8cdzw"]
Sep 08 20:38:02.364474 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:02.364438    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91368725-17db-4b56-b1bf-361151132ca7" path="/var/lib/kubelet/pods/91368725-17db-4b56-b1bf-361151132ca7/volumes"
Sep 08 20:38:17.383601 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.383563    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"]
Sep 08 20:38:17.384806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.384774    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91368725-17db-4b56-b1bf-361151132ca7" containerName="authorino"
Sep 08 20:38:17.384806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.384806    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="91368725-17db-4b56-b1bf-361151132ca7" containerName="authorino"
Sep 08 20:38:17.385041 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.384955    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="91368725-17db-4b56-b1bf-361151132ca7" containerName="authorino"
Sep 08 20:38:17.389794 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.389763    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.392341 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.392318    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6f8575-openshift-default-dockercfg-sj66r\""
Sep 08 20:38:17.398973 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.398252    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"]
Sep 08 20:38:17.508912 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.508853    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-podinfo\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.509105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.508921    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-envoy\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.509105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.508970    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-data\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.509105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.508984    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istiod-ca-cert\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.509105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.509046    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc6mt\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-kube-api-access-dc6mt\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.509105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.509099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-certs\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.509311 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.509116    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-token\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.509311 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.509138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-socket\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.509311 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.509241    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-credential-socket\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610110    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-data\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610153    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istiod-ca-cert\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610175    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc6mt\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-kube-api-access-dc6mt\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610212    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-certs\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610230    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-token\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610250    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-socket\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610329    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-credential-socket\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-podinfo\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610418    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-envoy\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610595    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-data\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610646    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-certs\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.610876 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610842    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-socket\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.611038 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.610883    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-credential-socket\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.611148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.611126    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istiod-ca-cert\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.612965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.612945    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-envoy\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.613111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.613091    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-podinfo\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.619835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.619807    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-token\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.620027 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.619955    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc6mt\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-kube-api-access-dc6mt\") pod \"e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") " pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.704305 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.704258    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:17.987824 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.987796    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"]
Sep 08 20:38:17.989757 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:17.989728    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2c5da4_44f0_4008_b3e8_6cfce2784dfc.slice/crio-57d1e80f22cc558bcd395ea9994f40ae9ffb4e572fd1fc7d11803ab766e426e5 WatchSource:0}: Error finding container 57d1e80f22cc558bcd395ea9994f40ae9ffb4e572fd1fc7d11803ab766e426e5: Status 404 returned error can't find the container with id 57d1e80f22cc558bcd395ea9994f40ae9ffb4e572fd1fc7d11803ab766e426e5
Sep 08 20:38:17.992108 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.992076    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:38:17.992207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.992173    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:38:17.992251 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:17.992206    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:38:18.727716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:18.727664    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" event={"ID":"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc","Type":"ContainerStarted","Data":"2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb"}
Sep 08 20:38:18.728145 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:18.727733    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" event={"ID":"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc","Type":"ContainerStarted","Data":"57d1e80f22cc558bcd395ea9994f40ae9ffb4e572fd1fc7d11803ab766e426e5"}
Sep 08 20:38:18.748749 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:18.748683    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" podStartSLOduration=1.7486604510000001 podStartE2EDuration="1.748660451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:18.748198673 +0000 UTC m=+784.913369249" watchObservedRunningTime="2026-09-08 20:38:18.748660451 +0000 UTC m=+784.913831029"
Sep 08 20:38:18.992550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:18.992461    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7713-140411532268976"]
Sep 08 20:38:18.996207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:18.996188    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7713-140411532268976"
Sep 08 20:38:18.999217 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:18.999191    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qqm77\""
Sep 08 20:38:19.001761 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.001736    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7713-140411532268976"]
Sep 08 20:38:19.023536 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.023499    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d5lb\" (UniqueName: \"kubernetes.io/projected/6cbb5d39-8612-44e8-99db-3c850a17c6d8-kube-api-access-6d5lb\") pod \"test-curl-7713-140411532268976\" (UID: \"6cbb5d39-8612-44e8-99db-3c850a17c6d8\") " pod="opendatahub/test-curl-7713-140411532268976"
Sep 08 20:38:19.124501 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.124461    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d5lb\" (UniqueName: \"kubernetes.io/projected/6cbb5d39-8612-44e8-99db-3c850a17c6d8-kube-api-access-6d5lb\") pod \"test-curl-7713-140411532268976\" (UID: \"6cbb5d39-8612-44e8-99db-3c850a17c6d8\") " pod="opendatahub/test-curl-7713-140411532268976"
Sep 08 20:38:19.134221 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.134186    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d5lb\" (UniqueName: \"kubernetes.io/projected/6cbb5d39-8612-44e8-99db-3c850a17c6d8-kube-api-access-6d5lb\") pod \"test-curl-7713-140411532268976\" (UID: \"6cbb5d39-8612-44e8-99db-3c850a17c6d8\") " pod="opendatahub/test-curl-7713-140411532268976"
Sep 08 20:38:19.307144 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.307054    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7713-140411532268976"
Sep 08 20:38:19.587570 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.587545    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7713-140411532268976"]
Sep 08 20:38:19.589299 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:19.589271    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cbb5d39_8612_44e8_99db_3c850a17c6d8.slice/crio-4c5883e98c157ff738e97b414a9dc6e3202c2ddd8f0f859537c786da85c03d8f WatchSource:0}: Error finding container 4c5883e98c157ff738e97b414a9dc6e3202c2ddd8f0f859537c786da85c03d8f: Status 404 returned error can't find the container with id 4c5883e98c157ff738e97b414a9dc6e3202c2ddd8f0f859537c786da85c03d8f
Sep 08 20:38:19.705458 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.705419    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:19.710603 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.710575    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:19.736543 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.736507    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7713-140411532268976" event={"ID":"6cbb5d39-8612-44e8-99db-3c850a17c6d8","Type":"ContainerStarted","Data":"4c5883e98c157ff738e97b414a9dc6e3202c2ddd8f0f859537c786da85c03d8f"}
Sep 08 20:38:19.736965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.736946    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:19.737888 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:19.737866    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:20.744795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:20.744757    2561 generic.go:356] "Generic (PLEG): container finished" podID="6cbb5d39-8612-44e8-99db-3c850a17c6d8" containerID="0e0e5f5115b0acc36414de313b570ebb95328f78faff9757bcfeb7a2a3c20f00" exitCode=0
Sep 08 20:38:20.745207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:20.744846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7713-140411532268976" event={"ID":"6cbb5d39-8612-44e8-99db-3c850a17c6d8","Type":"ContainerDied","Data":"0e0e5f5115b0acc36414de313b570ebb95328f78faff9757bcfeb7a2a3c20f00"}
Sep 08 20:38:21.904598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:21.904572    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7713-140411532268976"
Sep 08 20:38:21.940284 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:21.940254    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7713-140411532268976_6cbb5d39-8612-44e8-99db-3c850a17c6d8/test-curl-7713-140411532268976/0.log"
Sep 08 20:38:21.953100 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:21.953058    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d5lb\" (UniqueName: \"kubernetes.io/projected/6cbb5d39-8612-44e8-99db-3c850a17c6d8-kube-api-access-6d5lb\") pod \"6cbb5d39-8612-44e8-99db-3c850a17c6d8\" (UID: \"6cbb5d39-8612-44e8-99db-3c850a17c6d8\") "
Sep 08 20:38:21.955463 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:21.955416    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbb5d39-8612-44e8-99db-3c850a17c6d8-kube-api-access-6d5lb" (OuterVolumeSpecName: "kube-api-access-6d5lb") pod "6cbb5d39-8612-44e8-99db-3c850a17c6d8" (UID: "6cbb5d39-8612-44e8-99db-3c850a17c6d8"). InnerVolumeSpecName "kube-api-access-6d5lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:38:21.965654 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:21.965625    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7713-140411532268976"]
Sep 08 20:38:21.969117 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:21.969093    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7713-140411532268976"]
Sep 08 20:38:22.054246 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.054161    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d5lb\" (UniqueName: \"kubernetes.io/projected/6cbb5d39-8612-44e8-99db-3c850a17c6d8-kube-api-access-6d5lb\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:38:22.094335 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.094297    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7713-140411532271024"]
Sep 08 20:38:22.094815 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.094798    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbb5d39-8612-44e8-99db-3c850a17c6d8" containerName="test-curl-7713-140411532268976"
Sep 08 20:38:22.094963 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.094820    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbb5d39-8612-44e8-99db-3c850a17c6d8" containerName="test-curl-7713-140411532268976"
Sep 08 20:38:22.094963 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.094944    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cbb5d39-8612-44e8-99db-3c850a17c6d8" containerName="test-curl-7713-140411532268976"
Sep 08 20:38:22.098098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.098076    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7713-140411532271024"
Sep 08 20:38:22.109584 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.109558    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7713-140411532271024"]
Sep 08 20:38:22.155395 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.155361    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lv7l\" (UniqueName: \"kubernetes.io/projected/69ee28f0-f05e-4c89-b5e7-97fdb26abca7-kube-api-access-5lv7l\") pod \"test-curl-7713-140411532271024\" (UID: \"69ee28f0-f05e-4c89-b5e7-97fdb26abca7\") " pod="opendatahub/test-curl-7713-140411532271024"
Sep 08 20:38:22.256573 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.256534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lv7l\" (UniqueName: \"kubernetes.io/projected/69ee28f0-f05e-4c89-b5e7-97fdb26abca7-kube-api-access-5lv7l\") pod \"test-curl-7713-140411532271024\" (UID: \"69ee28f0-f05e-4c89-b5e7-97fdb26abca7\") " pod="opendatahub/test-curl-7713-140411532271024"
Sep 08 20:38:22.294309 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.294268    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lv7l\" (UniqueName: \"kubernetes.io/projected/69ee28f0-f05e-4c89-b5e7-97fdb26abca7-kube-api-access-5lv7l\") pod \"test-curl-7713-140411532271024\" (UID: \"69ee28f0-f05e-4c89-b5e7-97fdb26abca7\") " pod="opendatahub/test-curl-7713-140411532271024"
Sep 08 20:38:22.364491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.364410    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cbb5d39-8612-44e8-99db-3c850a17c6d8" path="/var/lib/kubelet/pods/6cbb5d39-8612-44e8-99db-3c850a17c6d8/volumes"
Sep 08 20:38:22.408011 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.407977    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7713-140411532271024"
Sep 08 20:38:22.685361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.685326    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7713-140411532271024"]
Sep 08 20:38:22.687025 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:22.686998    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69ee28f0_f05e_4c89_b5e7_97fdb26abca7.slice/crio-8f7da49d8373be12cff8403ac47120af6e6cefc52bebe74b62e6b5ec7c5144fa WatchSource:0}: Error finding container 8f7da49d8373be12cff8403ac47120af6e6cefc52bebe74b62e6b5ec7c5144fa: Status 404 returned error can't find the container with id 8f7da49d8373be12cff8403ac47120af6e6cefc52bebe74b62e6b5ec7c5144fa
Sep 08 20:38:22.769010 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.768975    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7713-140411532268976"
Sep 08 20:38:22.769404 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.769344    2561 scope.go:117] "RemoveContainer" containerID="0e0e5f5115b0acc36414de313b570ebb95328f78faff9757bcfeb7a2a3c20f00"
Sep 08 20:38:22.773073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.773043    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7713-140411532271024" event={"ID":"69ee28f0-f05e-4c89-b5e7-97fdb26abca7","Type":"ContainerStarted","Data":"8f7da49d8373be12cff8403ac47120af6e6cefc52bebe74b62e6b5ec7c5144fa"}
Sep 08 20:38:22.851221 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.851185    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"]
Sep 08 20:38:22.857104 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.856954    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:22.865923 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.864330    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"]
Sep 08 20:38:22.876883 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.876848    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6f8575-dockercfg-xwrcw\""
Sep 08 20:38:22.878016 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.877109    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6f8575\""
Sep 08 20:38:22.888524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.888488    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"]
Sep 08 20:38:22.892979 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.892953    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:22.901783 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.901759    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"]
Sep 08 20:38:22.964595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.964466    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063d29a9-2ba8-47f8-a8f5-46cd485e897c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85\" (UID: \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:22.964979 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.964580    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dcb364f-e473-4c93-9a87-039c9902841a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6\" (UID: \"2dcb364f-e473-4c93-9a87-039c9902841a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:22.964979 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.964726    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnbqm\" (UniqueName: \"kubernetes.io/projected/063d29a9-2ba8-47f8-a8f5-46cd485e897c-kube-api-access-fnbqm\") pod \"payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85\" (UID: \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:22.964979 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:22.964839    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dhcf\" (UniqueName: \"kubernetes.io/projected/2dcb364f-e473-4c93-9a87-039c9902841a-kube-api-access-5dhcf\") pod \"payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6\" (UID: \"2dcb364f-e473-4c93-9a87-039c9902841a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:23.065835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.065783    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dhcf\" (UniqueName: \"kubernetes.io/projected/2dcb364f-e473-4c93-9a87-039c9902841a-kube-api-access-5dhcf\") pod \"payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6\" (UID: \"2dcb364f-e473-4c93-9a87-039c9902841a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:23.066080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.065954    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063d29a9-2ba8-47f8-a8f5-46cd485e897c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85\" (UID: \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:23.066080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.065990    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dcb364f-e473-4c93-9a87-039c9902841a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6\" (UID: \"2dcb364f-e473-4c93-9a87-039c9902841a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:23.066080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.066066    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnbqm\" (UniqueName: \"kubernetes.io/projected/063d29a9-2ba8-47f8-a8f5-46cd485e897c-kube-api-access-fnbqm\") pod \"payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85\" (UID: \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:23.066749 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.066726    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dcb364f-e473-4c93-9a87-039c9902841a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6\" (UID: \"2dcb364f-e473-4c93-9a87-039c9902841a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:23.066947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.066923    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063d29a9-2ba8-47f8-a8f5-46cd485e897c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85\" (UID: \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:23.075693 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.075660    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnbqm\" (UniqueName: \"kubernetes.io/projected/063d29a9-2ba8-47f8-a8f5-46cd485e897c-kube-api-access-fnbqm\") pod \"payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85\" (UID: \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:23.076321 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.076288    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dhcf\" (UniqueName: \"kubernetes.io/projected/2dcb364f-e473-4c93-9a87-039c9902841a-kube-api-access-5dhcf\") pod \"payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6\" (UID: \"2dcb364f-e473-4c93-9a87-039c9902841a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:23.208348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.208306    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:23.215054 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.214984    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:23.492517 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:23.492483    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dcb364f_e473_4c93_9a87_039c9902841a.slice/crio-bfab9ca1a317fd25ce4998f7acb8834c6dec0a4088e186db1a842c3075699b2e WatchSource:0}: Error finding container bfab9ca1a317fd25ce4998f7acb8834c6dec0a4088e186db1a842c3075699b2e: Status 404 returned error can't find the container with id bfab9ca1a317fd25ce4998f7acb8834c6dec0a4088e186db1a842c3075699b2e
Sep 08 20:38:23.492757 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.492733    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"]
Sep 08 20:38:23.506407 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.506384    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"]
Sep 08 20:38:23.510102 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:23.510071    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063d29a9_2ba8_47f8_a8f5_46cd485e897c.slice/crio-fc1f0cb71ef41341a4dd38adf68d43157828a7df5387ff6a0a2ee02f90302c08 WatchSource:0}: Error finding container fc1f0cb71ef41341a4dd38adf68d43157828a7df5387ff6a0a2ee02f90302c08: Status 404 returned error can't find the container with id fc1f0cb71ef41341a4dd38adf68d43157828a7df5387ff6a0a2ee02f90302c08
Sep 08 20:38:23.783603 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.783575    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85" event={"ID":"063d29a9-2ba8-47f8-a8f5-46cd485e897c","Type":"ContainerStarted","Data":"28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07"}
Sep 08 20:38:23.783751 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.783729    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85" event={"ID":"063d29a9-2ba8-47f8-a8f5-46cd485e897c","Type":"ContainerStarted","Data":"fc1f0cb71ef41341a4dd38adf68d43157828a7df5387ff6a0a2ee02f90302c08"}
Sep 08 20:38:23.785243 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.785218    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6" event={"ID":"2dcb364f-e473-4c93-9a87-039c9902841a","Type":"ContainerStarted","Data":"8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec"}
Sep 08 20:38:23.785368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.785266    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6" event={"ID":"2dcb364f-e473-4c93-9a87-039c9902841a","Type":"ContainerStarted","Data":"bfab9ca1a317fd25ce4998f7acb8834c6dec0a4088e186db1a842c3075699b2e"}
Sep 08 20:38:23.785368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.785313    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:23.819794 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.819730    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85" podStartSLOduration=1.819703538 podStartE2EDuration="1.819703538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:23.817484619 +0000 UTC m=+789.982655196" watchObservedRunningTime="2026-09-08 20:38:23.819703538 +0000 UTC m=+789.984874115"
Sep 08 20:38:23.854966 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:23.854887    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6" podStartSLOduration=1.854865776 podStartE2EDuration="1.854865776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:23.852942107 +0000 UTC m=+790.018112684" watchObservedRunningTime="2026-09-08 20:38:23.854865776 +0000 UTC m=+790.020036356"
Sep 08 20:38:24.794194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:24.794161    2561 generic.go:356] "Generic (PLEG): container finished" podID="69ee28f0-f05e-4c89-b5e7-97fdb26abca7" containerID="cce0232250c79ab51a73d6ececb2fb4f58b8d3bfdc1f9cac2898aea5767bf471" exitCode=0
Sep 08 20:38:24.794625 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:24.794251    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7713-140411532271024" event={"ID":"69ee28f0-f05e-4c89-b5e7-97fdb26abca7","Type":"ContainerDied","Data":"cce0232250c79ab51a73d6ececb2fb4f58b8d3bfdc1f9cac2898aea5767bf471"}
Sep 08 20:38:24.794733 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:24.794712    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:25.955029 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:25.955004    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7713-140411532271024"
Sep 08 20:38:25.978266 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:25.978236    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7713-140411532271024_69ee28f0-f05e-4c89-b5e7-97fdb26abca7/test-curl-7713-140411532271024/0.log"
Sep 08 20:38:26.001392 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:26.001360    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7713-140411532271024"]
Sep 08 20:38:26.003925 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:26.003904    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7713-140411532271024"]
Sep 08 20:38:26.099257 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:26.099158    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lv7l\" (UniqueName: \"kubernetes.io/projected/69ee28f0-f05e-4c89-b5e7-97fdb26abca7-kube-api-access-5lv7l\") pod \"69ee28f0-f05e-4c89-b5e7-97fdb26abca7\" (UID: \"69ee28f0-f05e-4c89-b5e7-97fdb26abca7\") "
Sep 08 20:38:26.101301 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:26.101269    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ee28f0-f05e-4c89-b5e7-97fdb26abca7-kube-api-access-5lv7l" (OuterVolumeSpecName: "kube-api-access-5lv7l") pod "69ee28f0-f05e-4c89-b5e7-97fdb26abca7" (UID: "69ee28f0-f05e-4c89-b5e7-97fdb26abca7"). InnerVolumeSpecName "kube-api-access-5lv7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:38:26.200237 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:26.200199    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lv7l\" (UniqueName: \"kubernetes.io/projected/69ee28f0-f05e-4c89-b5e7-97fdb26abca7-kube-api-access-5lv7l\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:38:26.363710 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:26.363632    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ee28f0-f05e-4c89-b5e7-97fdb26abca7" path="/var/lib/kubelet/pods/69ee28f0-f05e-4c89-b5e7-97fdb26abca7/volumes"
Sep 08 20:38:26.414751 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:38:26.414711    2561 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69ee28f0_f05e_4c89_b5e7_97fdb26abca7.slice\": RecentStats: unable to find data in memory cache]"
Sep 08 20:38:26.811331 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:26.811297    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7713-140411532271024"
Sep 08 20:38:26.811526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:26.811312    2561 scope.go:117] "RemoveContainer" containerID="cce0232250c79ab51a73d6ececb2fb4f58b8d3bfdc1f9cac2898aea5767bf471"
Sep 08 20:38:30.141975 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.141932    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"]
Sep 08 20:38:30.142644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.142619    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ee28f0-f05e-4c89-b5e7-97fdb26abca7" containerName="test-curl-7713-140411532271024"
Sep 08 20:38:30.142644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.142646    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ee28f0-f05e-4c89-b5e7-97fdb26abca7" containerName="test-curl-7713-140411532271024"
Sep 08 20:38:30.142796 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.142780    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ee28f0-f05e-4c89-b5e7-97fdb26abca7" containerName="test-curl-7713-140411532271024"
Sep 08 20:38:30.148212 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.148180    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.151166 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.151139    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z8244\""
Sep 08 20:38:30.160215 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.160170    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"]
Sep 08 20:38:30.162621 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.162377    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 20:38:30.165489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.165456    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c73d714b-serving-cert\""
Sep 08 20:38:30.242591 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.242548    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvxb5\" (UniqueName: \"kubernetes.io/projected/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-kube-api-access-gvxb5\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.242940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.242889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-maas-api-tls\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.243149 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.243124    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-metrics-certs\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.243361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.243343    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-openshift-service-ca\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.344709 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.344661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-maas-api-tls\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.344953 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.344735    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-metrics-certs\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.344953 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.344819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-openshift-service-ca\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.344953 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.344935    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvxb5\" (UniqueName: \"kubernetes.io/projected/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-kube-api-access-gvxb5\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.346283 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.346045    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-openshift-service-ca\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.351557 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.349501    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-maas-api-tls\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.363408 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.361413    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvxb5\" (UniqueName: \"kubernetes.io/projected/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-kube-api-access-gvxb5\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.364207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.364176    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-metrics-certs\") pod \"maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.476273 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.476227    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:30.796481 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.796402    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"]
Sep 08 20:38:30.799218 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:30.799190    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f4e8fc6_f2ee_4038_98cd_6a6c90d75bd4.slice/crio-6751217a603062e35d92edbb821b80fc70a58b4f5c3bff2a454921bd44f0aad9 WatchSource:0}: Error finding container 6751217a603062e35d92edbb821b80fc70a58b4f5c3bff2a454921bd44f0aad9: Status 404 returned error can't find the container with id 6751217a603062e35d92edbb821b80fc70a58b4f5c3bff2a454921bd44f0aad9
Sep 08 20:38:30.860920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.860399    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q" event={"ID":"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4","Type":"ContainerStarted","Data":"6751217a603062e35d92edbb821b80fc70a58b4f5c3bff2a454921bd44f0aad9"}
Sep 08 20:38:30.971470 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.971439    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"]
Sep 08 20:38:30.976379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.976350    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:30.979187 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.979153    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c73d714b-dockercfg-m27kp\""
Sep 08 20:38:30.979330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.979198    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c73d714b\""
Sep 08 20:38:30.991295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:30.991265    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"]
Sep 08 20:38:31.064202 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.064113    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhcmc\" (UniqueName: \"kubernetes.io/projected/84b661d0-7b8e-40b8-b54a-d78338e15a6c-kube-api-access-jhcmc\") pod \"payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9\" (UID: \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\") " pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:31.064202 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.064159    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b661d0-7b8e-40b8-b54a-d78338e15a6c-plugins-config-volume\") pod \"payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9\" (UID: \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\") " pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:31.165732 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.165682    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhcmc\" (UniqueName: \"kubernetes.io/projected/84b661d0-7b8e-40b8-b54a-d78338e15a6c-kube-api-access-jhcmc\") pod \"payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9\" (UID: \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\") " pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:31.166160 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.165752    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b661d0-7b8e-40b8-b54a-d78338e15a6c-plugins-config-volume\") pod \"payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9\" (UID: \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\") " pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:31.166877 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.166852    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b661d0-7b8e-40b8-b54a-d78338e15a6c-plugins-config-volume\") pod \"payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9\" (UID: \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\") " pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:31.174155 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.174123    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhcmc\" (UniqueName: \"kubernetes.io/projected/84b661d0-7b8e-40b8-b54a-d78338e15a6c-kube-api-access-jhcmc\") pod \"payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9\" (UID: \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\") " pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:31.290173 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.290123    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:31.610974 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.610932    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"]
Sep 08 20:38:31.612752 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:31.612699    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b661d0_7b8e_40b8_b54a_d78338e15a6c.slice/crio-fbd22ead49a0d85c39491962a53779ee0eafc743fa59368c8d9e4567d3599c20 WatchSource:0}: Error finding container fbd22ead49a0d85c39491962a53779ee0eafc743fa59368c8d9e4567d3599c20: Status 404 returned error can't find the container with id fbd22ead49a0d85c39491962a53779ee0eafc743fa59368c8d9e4567d3599c20
Sep 08 20:38:31.876823 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.876046    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9" event={"ID":"84b661d0-7b8e-40b8-b54a-d78338e15a6c","Type":"ContainerStarted","Data":"8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47"}
Sep 08 20:38:31.876823 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.876151    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9" event={"ID":"84b661d0-7b8e-40b8-b54a-d78338e15a6c","Type":"ContainerStarted","Data":"fbd22ead49a0d85c39491962a53779ee0eafc743fa59368c8d9e4567d3599c20"}
Sep 08 20:38:31.876823 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.876367    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:31.895768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:31.895715    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9" podStartSLOduration=1.8956977 podStartE2EDuration="1.8956977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:31.894844596 +0000 UTC m=+798.060015173" watchObservedRunningTime="2026-09-08 20:38:31.8956977 +0000 UTC m=+798.060868288"
Sep 08 20:38:32.603572 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.603529    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"]
Sep 08 20:38:32.608021 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.607993    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:32.613693 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.613661    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-91d46f\""
Sep 08 20:38:32.613914 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.613880    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-91d46f-dockercfg-bzd9w\""
Sep 08 20:38:32.618850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.618790    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"]
Sep 08 20:38:32.622831 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.622805    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"]
Sep 08 20:38:32.628074 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.628042    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:32.667573 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.667470    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"]
Sep 08 20:38:32.786473 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.786436    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b126292e-2a41-4199-9990-aade94b24353-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn\" (UID: \"b126292e-2a41-4199-9990-aade94b24353\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:32.786678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.786494    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72690696-02a1-4e86-ae68-4514dff67eba-plugins-config-volume\") pod \"payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc\" (UID: \"72690696-02a1-4e86-ae68-4514dff67eba\") " pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:32.786678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.786582    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkdt9\" (UniqueName: \"kubernetes.io/projected/72690696-02a1-4e86-ae68-4514dff67eba-kube-api-access-bkdt9\") pod \"payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc\" (UID: \"72690696-02a1-4e86-ae68-4514dff67eba\") " pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:32.786678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.786642    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b5vf\" (UniqueName: \"kubernetes.io/projected/b126292e-2a41-4199-9990-aade94b24353-kube-api-access-6b5vf\") pod \"payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn\" (UID: \"b126292e-2a41-4199-9990-aade94b24353\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:32.888200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.888127    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b126292e-2a41-4199-9990-aade94b24353-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn\" (UID: \"b126292e-2a41-4199-9990-aade94b24353\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:32.888200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.888163    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72690696-02a1-4e86-ae68-4514dff67eba-plugins-config-volume\") pod \"payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc\" (UID: \"72690696-02a1-4e86-ae68-4514dff67eba\") " pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:32.888200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.888185    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkdt9\" (UniqueName: \"kubernetes.io/projected/72690696-02a1-4e86-ae68-4514dff67eba-kube-api-access-bkdt9\") pod \"payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc\" (UID: \"72690696-02a1-4e86-ae68-4514dff67eba\") " pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:32.888483 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.888222    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b5vf\" (UniqueName: \"kubernetes.io/projected/b126292e-2a41-4199-9990-aade94b24353-kube-api-access-6b5vf\") pod \"payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn\" (UID: \"b126292e-2a41-4199-9990-aade94b24353\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:32.888811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.888782    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b126292e-2a41-4199-9990-aade94b24353-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn\" (UID: \"b126292e-2a41-4199-9990-aade94b24353\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:32.888939 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.888890    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72690696-02a1-4e86-ae68-4514dff67eba-plugins-config-volume\") pod \"payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc\" (UID: \"72690696-02a1-4e86-ae68-4514dff67eba\") " pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:32.897955 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.897878    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b5vf\" (UniqueName: \"kubernetes.io/projected/b126292e-2a41-4199-9990-aade94b24353-kube-api-access-6b5vf\") pod \"payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn\" (UID: \"b126292e-2a41-4199-9990-aade94b24353\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:32.898125 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.898008    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkdt9\" (UniqueName: \"kubernetes.io/projected/72690696-02a1-4e86-ae68-4514dff67eba-kube-api-access-bkdt9\") pod \"payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc\" (UID: \"72690696-02a1-4e86-ae68-4514dff67eba\") " pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:32.926382 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.926160    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:32.969610 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:32.969369    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:33.228454 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.228344    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"]
Sep 08 20:38:33.230917 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:33.230859    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb126292e_2a41_4199_9990_aade94b24353.slice/crio-c183bba850e2d98762fbea1ed202bc444ae199117149799e26ebb789ee27ccd4 WatchSource:0}: Error finding container c183bba850e2d98762fbea1ed202bc444ae199117149799e26ebb789ee27ccd4: Status 404 returned error can't find the container with id c183bba850e2d98762fbea1ed202bc444ae199117149799e26ebb789ee27ccd4
Sep 08 20:38:33.290596 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.290578    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"]
Sep 08 20:38:33.295348 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:38:33.295313    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72690696_02a1_4e86_ae68_4514dff67eba.slice/crio-0824a721ac37e83154315b7e71e92792c9340187eab48ea37e9012110ceb71da WatchSource:0}: Error finding container 0824a721ac37e83154315b7e71e92792c9340187eab48ea37e9012110ceb71da: Status 404 returned error can't find the container with id 0824a721ac37e83154315b7e71e92792c9340187eab48ea37e9012110ceb71da
Sep 08 20:38:33.893725 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.893685    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q" event={"ID":"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4","Type":"ContainerStarted","Data":"7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790"}
Sep 08 20:38:33.894214 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.893753    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:33.895593 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.895520    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc" event={"ID":"72690696-02a1-4e86-ae68-4514dff67eba","Type":"ContainerStarted","Data":"f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16"}
Sep 08 20:38:33.895593 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.895564    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc" event={"ID":"72690696-02a1-4e86-ae68-4514dff67eba","Type":"ContainerStarted","Data":"0824a721ac37e83154315b7e71e92792c9340187eab48ea37e9012110ceb71da"}
Sep 08 20:38:33.897167 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.897148    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn" event={"ID":"b126292e-2a41-4199-9990-aade94b24353","Type":"ContainerStarted","Data":"481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02"}
Sep 08 20:38:33.897277 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.897199    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn" event={"ID":"b126292e-2a41-4199-9990-aade94b24353","Type":"ContainerStarted","Data":"c183bba850e2d98762fbea1ed202bc444ae199117149799e26ebb789ee27ccd4"}
Sep 08 20:38:33.897277 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.897230    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:33.915290 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.915244    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q" podStartSLOduration=1.788509218 podStartE2EDuration="3.915226682s" totalImagesPullingTime="2.126717464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:33.913413411 +0000 UTC m=+800.078583986" watchObservedRunningTime="2026-09-08 20:38:33.915226682 +0000 UTC m=+800.080397259"
Sep 08 20:38:33.933161 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.933000    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc" podStartSLOduration=1.932981595 podStartE2EDuration="1.932981595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:33.92930155 +0000 UTC m=+800.094472123" watchObservedRunningTime="2026-09-08 20:38:33.932981595 +0000 UTC m=+800.098152174"
Sep 08 20:38:33.947877 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:33.947810    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn" podStartSLOduration=1.947788009 podStartE2EDuration="1.947788009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:33.94521696 +0000 UTC m=+800.110387549" watchObservedRunningTime="2026-09-08 20:38:33.947788009 +0000 UTC m=+800.112958585"
Sep 08 20:38:34.797096 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:34.797060    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:38:34.904457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:34.904413    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:38:35.803148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:35.803046    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:38:38.932590 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:38.932500    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerID="2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb" exitCode=0
Sep 08 20:38:38.933219 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:38.932582    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" event={"ID":"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc","Type":"ContainerDied","Data":"2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb"}
Sep 08 20:38:38.933219 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:38.933161    2561 scope.go:117] "RemoveContainer" containerID="2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb"
Sep 08 20:38:38.934552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:38.934524    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:38:38.934678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:38.934612    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:38:38.934678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:38.934660    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:38:39.910937 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:39.910886    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:38:39.958561 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:39.956383    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" event={"ID":"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc","Type":"ContainerStarted","Data":"b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088"}
Sep 08 20:38:40.704685 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:40.704649    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:40.704958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:40.704783    2561 patch_prober.go:28] interesting pod/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 08 20:38:40.704958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:40.704835    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 08 20:38:41.705323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:41.705285    2561 patch_prober.go:28] interesting pod/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 08 20:38:41.705767 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:41.705353    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 08 20:38:42.705226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:42.705196    2561 patch_prober.go:28] interesting pod/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 08 20:38:42.705509 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:42.705475    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 08 20:38:42.886884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:42.886853    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:38:43.705382 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:43.705344    2561 patch_prober.go:28] interesting pod/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 08 20:38:43.705562 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:43.705398    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 08 20:38:44.705225 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:44.705190    2561 patch_prober.go:28] interesting pod/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 08 20:38:44.705424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:44.705249    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 08 20:38:44.906553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:44.906521    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:38:45.708998 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:45.708964    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:45.709401 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:45.709377    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:45.710183 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:45.710163    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:38:45.912843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:38:45.912806    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:39:10.766200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:10.766160    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"]
Sep 08 20:39:10.766830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:10.766404    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q" podUID="7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" containerName="maas-api" containerID="cri-o://7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790" gracePeriod=30
Sep 08 20:39:10.832337 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:10.831244    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"]
Sep 08 20:39:10.832337 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:10.831489    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9" podUID="84b661d0-7b8e-40b8-b54a-d78338e15a6c" containerName="payload-processing" containerID="cri-o://8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47" gracePeriod=30
Sep 08 20:39:11.149493 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.149465    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:39:11.154059 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.154034    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:39:11.183543 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.183510    2561 generic.go:356] "Generic (PLEG): container finished" podID="7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" containerID="7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790" exitCode=0
Sep 08 20:39:11.183735 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.183566    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"
Sep 08 20:39:11.183735 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.183596    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q" event={"ID":"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4","Type":"ContainerDied","Data":"7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790"}
Sep 08 20:39:11.183735 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.183665    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q" event={"ID":"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4","Type":"ContainerDied","Data":"6751217a603062e35d92edbb821b80fc70a58b4f5c3bff2a454921bd44f0aad9"}
Sep 08 20:39:11.183735 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.183705    2561 scope.go:117] "RemoveContainer" containerID="7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790"
Sep 08 20:39:11.185275 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.185248    2561 generic.go:356] "Generic (PLEG): container finished" podID="84b661d0-7b8e-40b8-b54a-d78338e15a6c" containerID="8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47" exitCode=0
Sep 08 20:39:11.185423 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.185281    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9" event={"ID":"84b661d0-7b8e-40b8-b54a-d78338e15a6c","Type":"ContainerDied","Data":"8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47"}
Sep 08 20:39:11.185423 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.185340    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9" event={"ID":"84b661d0-7b8e-40b8-b54a-d78338e15a6c","Type":"ContainerDied","Data":"fbd22ead49a0d85c39491962a53779ee0eafc743fa59368c8d9e4567d3599c20"}
Sep 08 20:39:11.185423 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.185292    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"
Sep 08 20:39:11.195554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.195532    2561 scope.go:117] "RemoveContainer" containerID="7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790"
Sep 08 20:39:11.196035 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:39:11.195919    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790\": container with ID starting with 7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790 not found: ID does not exist" containerID="7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790"
Sep 08 20:39:11.196035 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.195961    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790"} err="failed to get container status \"7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790\": rpc error: code = NotFound desc = could not find container \"7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790\": container with ID starting with 7bed84f135b6f882be37f69cda30064227b828f436eb815404ba3c52b2971790 not found: ID does not exist"
Sep 08 20:39:11.196035 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.195986    2561 scope.go:117] "RemoveContainer" containerID="8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47"
Sep 08 20:39:11.207459 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.207384    2561 scope.go:117] "RemoveContainer" containerID="8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47"
Sep 08 20:39:11.207768 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:39:11.207742    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47\": container with ID starting with 8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47 not found: ID does not exist" containerID="8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47"
Sep 08 20:39:11.207835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.207780    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47"} err="failed to get container status \"8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47\": rpc error: code = NotFound desc = could not find container \"8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47\": container with ID starting with 8eb3801664071dcc355d90298bc538ccee34bb83fee943e0bf907095e5ad0a47 not found: ID does not exist"
Sep 08 20:39:11.285080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.284990    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhcmc\" (UniqueName: \"kubernetes.io/projected/84b661d0-7b8e-40b8-b54a-d78338e15a6c-kube-api-access-jhcmc\") pod \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\" (UID: \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\") "
Sep 08 20:39:11.285080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.285029    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvxb5\" (UniqueName: \"kubernetes.io/projected/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-kube-api-access-gvxb5\") pod \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") "
Sep 08 20:39:11.285322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.285083    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-openshift-service-ca\") pod \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") "
Sep 08 20:39:11.285322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.285131    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-metrics-certs\") pod \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") "
Sep 08 20:39:11.285322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.285150    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-maas-api-tls\") pod \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\" (UID: \"7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4\") "
Sep 08 20:39:11.285322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.285207    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b661d0-7b8e-40b8-b54a-d78338e15a6c-plugins-config-volume\") pod \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\" (UID: \"84b661d0-7b8e-40b8-b54a-d78338e15a6c\") "
Sep 08 20:39:11.285618 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.285586    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" (UID: "7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:39:11.286543 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.286508    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84b661d0-7b8e-40b8-b54a-d78338e15a6c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84b661d0-7b8e-40b8-b54a-d78338e15a6c" (UID: "84b661d0-7b8e-40b8-b54a-d78338e15a6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:39:11.287360 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.287329    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b661d0-7b8e-40b8-b54a-d78338e15a6c-kube-api-access-jhcmc" (OuterVolumeSpecName: "kube-api-access-jhcmc") pod "84b661d0-7b8e-40b8-b54a-d78338e15a6c" (UID: "84b661d0-7b8e-40b8-b54a-d78338e15a6c"). InnerVolumeSpecName "kube-api-access-jhcmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:39:11.287467 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.287413    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" (UID: "7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:39:11.287517 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.287492    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-kube-api-access-gvxb5" (OuterVolumeSpecName: "kube-api-access-gvxb5") pod "7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" (UID: "7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4"). InnerVolumeSpecName "kube-api-access-gvxb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:39:11.287517 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.287491    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" (UID: "7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:39:11.386308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.386272    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b661d0-7b8e-40b8-b54a-d78338e15a6c-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:11.386308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.386308    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhcmc\" (UniqueName: \"kubernetes.io/projected/84b661d0-7b8e-40b8-b54a-d78338e15a6c-kube-api-access-jhcmc\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:11.386519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.386322    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvxb5\" (UniqueName: \"kubernetes.io/projected/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-kube-api-access-gvxb5\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:11.386519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.386332    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-openshift-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:11.386519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.386341    2561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-metrics-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:11.386519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.386350    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4-maas-api-tls\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:11.511631 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.511591    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"]
Sep 08 20:39:11.515922 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.515879    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c73d714b-5467bfd688-xcs9q"]
Sep 08 20:39:11.525960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.525929    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"]
Sep 08 20:39:11.529806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:11.529779    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c73d714b-65578bf765-w5gl9"]
Sep 08 20:39:12.095617 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.095518    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"]
Sep 08 20:39:12.096028 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.096019    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" containerName="maas-api"
Sep 08 20:39:12.096073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.096030    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" containerName="maas-api"
Sep 08 20:39:12.096073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.096044    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84b661d0-7b8e-40b8-b54a-d78338e15a6c" containerName="payload-processing"
Sep 08 20:39:12.096073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.096050    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="84b661d0-7b8e-40b8-b54a-d78338e15a6c" containerName="payload-processing"
Sep 08 20:39:12.096174 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.096130    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" containerName="maas-api"
Sep 08 20:39:12.096174 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.096140    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="84b661d0-7b8e-40b8-b54a-d78338e15a6c" containerName="payload-processing"
Sep 08 20:39:12.099484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.099459    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.102092 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.102070    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-4d417ac0-gw-openshift-default-dockercfg-hchpm\""
Sep 08 20:39:12.114650 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.114623    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"]
Sep 08 20:39:12.195308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-data\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.195308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195321    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-certs\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.195580 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-podinfo\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.195580 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195457    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-token\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.195580 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195486    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cppr\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-kube-api-access-8cppr\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.195754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195644    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-credential-socket\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.195754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195716    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-envoy\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.195862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195762    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-socket\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.195862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.195788    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istiod-ca-cert\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.296837 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.296798    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-credential-socket\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.296837 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.296844    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-envoy\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.296878    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-socket\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.296924    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istiod-ca-cert\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.296968    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-data\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297000    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-certs\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297034    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-podinfo\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297078    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-token\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297108    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cppr\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-kube-api-access-8cppr\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297495 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297446    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-socket\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-credential-socket\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istiod-ca-cert\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297745    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-certs\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.297890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.297761    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-data\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.299472 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.299446    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-envoy\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.299571 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.299553    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-podinfo\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.308588 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.308560    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-token\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.308836 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.308812    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cppr\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-kube-api-access-8cppr\") pod \"e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") " pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.369277 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.369187    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4" path="/var/lib/kubelet/pods/7f4e8fc6-f2ee-4038-98cd-6a6c90d75bd4/volumes"
Sep 08 20:39:12.369840 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.369817    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84b661d0-7b8e-40b8-b54a-d78338e15a6c" path="/var/lib/kubelet/pods/84b661d0-7b8e-40b8-b54a-d78338e15a6c/volumes"
Sep 08 20:39:12.411666 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.411628    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:12.697754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.697721    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"]
Sep 08 20:39:12.698981 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:39:12.698952    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d6df25_0e7b_4cf6_b4da_4b84b98a5091.slice/crio-7e7b6cf58aead07efa67efafaa2d80727887b55e3b59dfa256c909a133467960 WatchSource:0}: Error finding container 7e7b6cf58aead07efa67efafaa2d80727887b55e3b59dfa256c909a133467960: Status 404 returned error can't find the container with id 7e7b6cf58aead07efa67efafaa2d80727887b55e3b59dfa256c909a133467960
Sep 08 20:39:12.701217 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.701183    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:12.701313 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.701251    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:12.701313 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:12.701278    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:13.209143 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:13.209100    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" event={"ID":"43d6df25-0e7b-4cf6-b4da-4b84b98a5091","Type":"ContainerStarted","Data":"e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28"}
Sep 08 20:39:13.209535 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:13.209161    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" event={"ID":"43d6df25-0e7b-4cf6-b4da-4b84b98a5091","Type":"ContainerStarted","Data":"7e7b6cf58aead07efa67efafaa2d80727887b55e3b59dfa256c909a133467960"}
Sep 08 20:39:13.230814 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:13.230759    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" podStartSLOduration=1.230743643 podStartE2EDuration="1.230743643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:39:13.228262587 +0000 UTC m=+839.393433163" watchObservedRunningTime="2026-09-08 20:39:13.230743643 +0000 UTC m=+839.395914219"
Sep 08 20:39:13.412722 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:13.412680    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:13.418316 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:13.418283    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:14.216154 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.216124    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:14.217348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.217323    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:14.791388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.791354    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"]
Sep 08 20:39:14.797315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.797287    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:14.799823 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.799797    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-4d417ac0-serving-cert\""
Sep 08 20:39:14.799982 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.799929    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z8244\""
Sep 08 20:39:14.800958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.800941    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 20:39:14.809126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.809099    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"]
Sep 08 20:39:14.921920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.921847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwq47\" (UniqueName: \"kubernetes.io/projected/cc50b4ae-4797-44a4-aa28-1ac01712136b-kube-api-access-lwq47\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:14.922110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.921937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-metrics-certs\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:14.922110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.921960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-maas-api-tls\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:14.922110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:14.922033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc50b4ae-4797-44a4-aa28-1ac01712136b-openshift-service-ca\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.022847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.022806    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwq47\" (UniqueName: \"kubernetes.io/projected/cc50b4ae-4797-44a4-aa28-1ac01712136b-kube-api-access-lwq47\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.023058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.022864    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-metrics-certs\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.023058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.022889    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-maas-api-tls\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.023058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.022985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc50b4ae-4797-44a4-aa28-1ac01712136b-openshift-service-ca\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.023625 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.023603    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc50b4ae-4797-44a4-aa28-1ac01712136b-openshift-service-ca\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.025650 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.025625    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-maas-api-tls\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.025771 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.025626    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-metrics-certs\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.033417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.033376    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwq47\" (UniqueName: \"kubernetes.io/projected/cc50b4ae-4797-44a4-aa28-1ac01712136b-kube-api-access-lwq47\") pod \"maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.110278 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.110193    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:15.414844 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:15.414813    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"]
Sep 08 20:39:15.415686 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:39:15.415657    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc50b4ae_4797_44a4_aa28_1ac01712136b.slice/crio-c368affbf1ce793faea4916fd5c54f5c822fe9d1d1c473ad4f8cd8499316fc6c WatchSource:0}: Error finding container c368affbf1ce793faea4916fd5c54f5c822fe9d1d1c473ad4f8cd8499316fc6c: Status 404 returned error can't find the container with id c368affbf1ce793faea4916fd5c54f5c822fe9d1d1c473ad4f8cd8499316fc6c
Sep 08 20:39:16.231567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:16.231518    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl" event={"ID":"cc50b4ae-4797-44a4-aa28-1ac01712136b","Type":"ContainerStarted","Data":"d094e49c063657054895ce670c5c4ba8f86f509c4b953c08c047c0e48e0d0257"}
Sep 08 20:39:16.231792 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:16.231601    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl" event={"ID":"cc50b4ae-4797-44a4-aa28-1ac01712136b","Type":"ContainerStarted","Data":"c368affbf1ce793faea4916fd5c54f5c822fe9d1d1c473ad4f8cd8499316fc6c"}
Sep 08 20:39:16.251310 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:16.251246    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl" podStartSLOduration=1.8289476759999999 podStartE2EDuration="2.251228615s" totalImagesPullingTime="422.280939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:39:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:39:16.248659818 +0000 UTC m=+842.413830395" watchObservedRunningTime="2026-09-08 20:39:16.251228615 +0000 UTC m=+842.416399190"
Sep 08 20:39:17.238970 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:17.238935    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:23.252457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:23.252424    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:36.367334 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:36.367236    2561 generic.go:356] "Generic (PLEG): container finished" podID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerID="e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28" exitCode=0
Sep 08 20:39:36.367334 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:36.367304    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" event={"ID":"43d6df25-0e7b-4cf6-b4da-4b84b98a5091","Type":"ContainerDied","Data":"e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28"}
Sep 08 20:39:36.368002 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:36.367852    2561 scope.go:117] "RemoveContainer" containerID="e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28"
Sep 08 20:39:36.369077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:36.369048    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:36.369166 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:36.369126    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:36.369166 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:36.369157    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:37.375738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:37.375703    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" event={"ID":"43d6df25-0e7b-4cf6-b4da-4b84b98a5091","Type":"ContainerStarted","Data":"98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82"}
Sep 08 20:39:37.411797 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:37.411754    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:37.412009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:37.411988    2561 patch_prober.go:28] interesting pod/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 08 20:39:37.412060 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:37.412035    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 08 20:39:38.412318 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:38.412272    2561 patch_prober.go:28] interesting pod/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 08 20:39:38.412801 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:38.412347    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 08 20:39:39.413030 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:39.412988    2561 patch_prober.go:28] interesting pod/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 08 20:39:39.413425 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:39.413077    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 08 20:39:40.412004 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:40.411968    2561 patch_prober.go:28] interesting pod/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 08 20:39:40.412193 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:40.412044    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 08 20:39:41.412803 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:41.412759    2561 patch_prober.go:28] interesting pod/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 08 20:39:41.413264 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:41.412820    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 08 20:39:42.412098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:42.412058    2561 patch_prober.go:28] interesting pod/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 08 20:39:42.412374 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:42.412113    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 08 20:39:43.416391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:43.416359    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:43.416805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:43.416783    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:43.417499 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:43.417480    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:46.217477 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.217432    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"]
Sep 08 20:39:46.218810 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.218220    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl" podUID="cc50b4ae-4797-44a4-aa28-1ac01712136b" containerName="maas-api" containerID="cri-o://d094e49c063657054895ce670c5c4ba8f86f509c4b953c08c047c0e48e0d0257" gracePeriod=30
Sep 08 20:39:46.439926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.439870    2561 generic.go:356] "Generic (PLEG): container finished" podID="cc50b4ae-4797-44a4-aa28-1ac01712136b" containerID="d094e49c063657054895ce670c5c4ba8f86f509c4b953c08c047c0e48e0d0257" exitCode=0
Sep 08 20:39:46.440218 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.440012    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl" event={"ID":"cc50b4ae-4797-44a4-aa28-1ac01712136b","Type":"ContainerDied","Data":"d094e49c063657054895ce670c5c4ba8f86f509c4b953c08c047c0e48e0d0257"}
Sep 08 20:39:46.493748 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.493720    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:46.589677 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.589643    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"]
Sep 08 20:39:46.589970 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.589845    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy" containerID="cri-o://98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82" gracePeriod=30
Sep 08 20:39:46.659253 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.656660    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-maas-api-tls\") pod \"cc50b4ae-4797-44a4-aa28-1ac01712136b\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") "
Sep 08 20:39:46.659253 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.656742    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc50b4ae-4797-44a4-aa28-1ac01712136b-openshift-service-ca\") pod \"cc50b4ae-4797-44a4-aa28-1ac01712136b\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") "
Sep 08 20:39:46.659253 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.656790    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-metrics-certs\") pod \"cc50b4ae-4797-44a4-aa28-1ac01712136b\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") "
Sep 08 20:39:46.659253 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.656830    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwq47\" (UniqueName: \"kubernetes.io/projected/cc50b4ae-4797-44a4-aa28-1ac01712136b-kube-api-access-lwq47\") pod \"cc50b4ae-4797-44a4-aa28-1ac01712136b\" (UID: \"cc50b4ae-4797-44a4-aa28-1ac01712136b\") "
Sep 08 20:39:46.659253 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.657888    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc50b4ae-4797-44a4-aa28-1ac01712136b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc50b4ae-4797-44a4-aa28-1ac01712136b" (UID: "cc50b4ae-4797-44a4-aa28-1ac01712136b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:39:46.662083 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.662043    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc50b4ae-4797-44a4-aa28-1ac01712136b" (UID: "cc50b4ae-4797-44a4-aa28-1ac01712136b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:39:46.671180 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.671129    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cc50b4ae-4797-44a4-aa28-1ac01712136b" (UID: "cc50b4ae-4797-44a4-aa28-1ac01712136b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:39:46.680283 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.679211    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc50b4ae-4797-44a4-aa28-1ac01712136b-kube-api-access-lwq47" (OuterVolumeSpecName: "kube-api-access-lwq47") pod "cc50b4ae-4797-44a4-aa28-1ac01712136b" (UID: "cc50b4ae-4797-44a4-aa28-1ac01712136b"). InnerVolumeSpecName "kube-api-access-lwq47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:39:46.758664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.758582    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-maas-api-tls\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:46.758664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.758610    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc50b4ae-4797-44a4-aa28-1ac01712136b-openshift-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:46.758664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.758621    2561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc50b4ae-4797-44a4-aa28-1ac01712136b-metrics-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:46.758664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:46.758630    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwq47\" (UniqueName: \"kubernetes.io/projected/cc50b4ae-4797-44a4-aa28-1ac01712136b-kube-api-access-lwq47\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:47.284697 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.284662    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"]
Sep 08 20:39:47.285394 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.285375    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc50b4ae-4797-44a4-aa28-1ac01712136b" containerName="maas-api"
Sep 08 20:39:47.285472 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.285397    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc50b4ae-4797-44a4-aa28-1ac01712136b" containerName="maas-api"
Sep 08 20:39:47.285511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.285492    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc50b4ae-4797-44a4-aa28-1ac01712136b" containerName="maas-api"
Sep 08 20:39:47.289156 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.289134    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.291955 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.291934    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b2d463b5-openshift-default-dockercfg-89hv9\""
Sep 08 20:39:47.304121 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.304090    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"]
Sep 08 20:39:47.364574 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istiod-ca-cert\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.364754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364585    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-podinfo\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.364754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364624    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-token\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.364754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364678    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-credential-socket\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.364754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364708    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-data\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.364754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364734    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-socket\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.364931 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364763    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-envoy\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.364931 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-certs\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.364931 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.364870    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf5p5\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-kube-api-access-vf5p5\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.448267 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.448234    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"
Sep 08 20:39:47.448267 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.448256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl" event={"ID":"cc50b4ae-4797-44a4-aa28-1ac01712136b","Type":"ContainerDied","Data":"c368affbf1ce793faea4916fd5c54f5c822fe9d1d1c473ad4f8cd8499316fc6c"}
Sep 08 20:39:47.448492 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.448319    2561 scope.go:117] "RemoveContainer" containerID="d094e49c063657054895ce670c5c4ba8f86f509c4b953c08c047c0e48e0d0257"
Sep 08 20:39:47.466053 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466027    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-certs\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466171 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466059    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf5p5\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-kube-api-access-vf5p5\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466171 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466113    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istiod-ca-cert\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466171 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466166    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-podinfo\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466331 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-token\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466331 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-credential-socket\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466331 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466290    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-data\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466502 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466341    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-socket\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466502 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466383    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-envoy\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466502 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466421    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-certs\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466660 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466628    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-credential-socket\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466701    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-socket\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466774 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466730    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-data\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.466920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.466882    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istiod-ca-cert\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.468775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.468753    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-envoy\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.469050 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.469033    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-podinfo\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.471205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.471183    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"]
Sep 08 20:39:47.475414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.475393    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4d417ac0-79fd7d87d-4h9cl"]
Sep 08 20:39:47.475801 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.475782    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-token\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.476000 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.475982    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf5p5\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-kube-api-access-vf5p5\") pod \"e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") " pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.603244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.603160    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:47.882758 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.882725    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"]
Sep 08 20:39:47.883597 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:39:47.883573    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ca3a29_7cd6_4530_a5e9_3a70fb02aea9.slice/crio-84412bb8471a1b1c3f83c9b96c84402cc6834e4c2036d10afcdaf1c579763ca1 WatchSource:0}: Error finding container 84412bb8471a1b1c3f83c9b96c84402cc6834e4c2036d10afcdaf1c579763ca1: Status 404 returned error can't find the container with id 84412bb8471a1b1c3f83c9b96c84402cc6834e4c2036d10afcdaf1c579763ca1
Sep 08 20:39:47.886103 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.886062    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:47.886194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.886166    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:47.886238 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:47.886205    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:39:48.367505 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:48.367462    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc50b4ae-4797-44a4-aa28-1ac01712136b" path="/var/lib/kubelet/pods/cc50b4ae-4797-44a4-aa28-1ac01712136b/volumes"
Sep 08 20:39:48.458294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:48.458257    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" event={"ID":"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9","Type":"ContainerStarted","Data":"cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415"}
Sep 08 20:39:48.458510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:48.458317    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" event={"ID":"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9","Type":"ContainerStarted","Data":"84412bb8471a1b1c3f83c9b96c84402cc6834e4c2036d10afcdaf1c579763ca1"}
Sep 08 20:39:48.482959 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:48.482882    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" podStartSLOduration=1.482862089 podStartE2EDuration="1.482862089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:39:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:39:48.480855098 +0000 UTC m=+874.646025676" watchObservedRunningTime="2026-09-08 20:39:48.482862089 +0000 UTC m=+874.648032664"
Sep 08 20:39:48.606380 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:48.604238    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:48.611953 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:48.611870    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:49.464619 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:49.464589    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:49.465742 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:49.465720    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:39:51.981194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:51.981128    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:52.116360 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116320    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-data\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.116532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116420    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-envoy\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.116532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116459    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cppr\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-kube-api-access-8cppr\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.116532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116507    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-token\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.116665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116544    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istiod-ca-cert\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.116665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116576    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-credential-socket\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.116665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116628    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-podinfo\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.116665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116629    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-data" (OuterVolumeSpecName: "istio-data") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:52.116861 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116669    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-certs\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.116861 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116708    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-socket\") pod \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\" (UID: \"43d6df25-0e7b-4cf6-b4da-4b84b98a5091\") "
Sep 08 20:39:52.117026 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.116982    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:52.117103 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.117071    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:52.117183 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.117163    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:52.117256 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.117209    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:39:52.117324 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.117307    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.117381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.117329    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.117381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.117340    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.117381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.117353    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.119009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.118979    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-token" (OuterVolumeSpecName: "istio-token") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:39:52.119009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.118977    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:39:52.119154 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.119134    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:52.119192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.119154    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-kube-api-access-8cppr" (OuterVolumeSpecName: "kube-api-access-8cppr") pod "43d6df25-0e7b-4cf6-b4da-4b84b98a5091" (UID: "43d6df25-0e7b-4cf6-b4da-4b84b98a5091"). InnerVolumeSpecName "kube-api-access-8cppr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:39:52.218174 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.218143    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.218174 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.218169    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cppr\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-kube-api-access-8cppr\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.218174 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.218178    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.218174 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.218186    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.218448 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.218193    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d6df25-0e7b-4cf6-b4da-4b84b98a5091-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:39:52.486212 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.486179    2561 generic.go:356] "Generic (PLEG): container finished" podID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerID="98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82" exitCode=0
Sep 08 20:39:52.486410 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.486218    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" event={"ID":"43d6df25-0e7b-4cf6-b4da-4b84b98a5091","Type":"ContainerDied","Data":"98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82"}
Sep 08 20:39:52.486410 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.486247    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"
Sep 08 20:39:52.486410 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.486301    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29" event={"ID":"43d6df25-0e7b-4cf6-b4da-4b84b98a5091","Type":"ContainerDied","Data":"7e7b6cf58aead07efa67efafaa2d80727887b55e3b59dfa256c909a133467960"}
Sep 08 20:39:52.486410 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.486332    2561 scope.go:117] "RemoveContainer" containerID="98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82"
Sep 08 20:39:52.496547 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.496526    2561 scope.go:117] "RemoveContainer" containerID="e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28"
Sep 08 20:39:52.505562 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.505535    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"]
Sep 08 20:39:52.506300 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.506246    2561 scope.go:117] "RemoveContainer" containerID="98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82"
Sep 08 20:39:52.506662 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:39:52.506626    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82\": container with ID starting with 98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82 not found: ID does not exist" containerID="98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82"
Sep 08 20:39:52.506760 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.506673    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82"} err="failed to get container status \"98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82\": rpc error: code = NotFound desc = could not find container \"98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82\": container with ID starting with 98edb9e2017d7be46e49784acc52a3fe269e41e47988001db543918f4138cd82 not found: ID does not exist"
Sep 08 20:39:52.506760 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.506693    2561 scope.go:117] "RemoveContainer" containerID="e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28"
Sep 08 20:39:52.507048 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:39:52.507015    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28\": container with ID starting with e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28 not found: ID does not exist" containerID="e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28"
Sep 08 20:39:52.507103 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.507043    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28"} err="failed to get container status \"e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28\": rpc error: code = NotFound desc = could not find container \"e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28\": container with ID starting with e62e71b7484833bfc7ba22d1a001a666948fd59e76cc6d3b213be665477caf28 not found: ID does not exist"
Sep 08 20:39:52.510335 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:52.510315    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-4d417ac0-gw-openshift-default-774f8c96b4-jdw29"]
Sep 08 20:39:54.364715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:39:54.364680    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" path="/var/lib/kubelet/pods/43d6df25-0e7b-4cf6-b4da-4b84b98a5091/volumes"
Sep 08 20:40:11.630362 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:11.630321    2561 generic.go:356] "Generic (PLEG): container finished" podID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerID="cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415" exitCode=0
Sep 08 20:40:11.630799 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:11.630385    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" event={"ID":"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9","Type":"ContainerDied","Data":"cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415"}
Sep 08 20:40:11.630954 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:11.630928    2561 scope.go:117] "RemoveContainer" containerID="cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415"
Sep 08 20:40:11.632332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:11.632299    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:11.632445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:11.632414    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:11.632511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:11.632465    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:12.643949 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:12.643905    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" event={"ID":"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9","Type":"ContainerStarted","Data":"faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d"}
Sep 08 20:40:13.360571 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.360528    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"]
Sep 08 20:40:13.361285 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.361255    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy"
Sep 08 20:40:13.361285 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.361282    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy"
Sep 08 20:40:13.361476 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.361296    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy"
Sep 08 20:40:13.361476 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.361305    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy"
Sep 08 20:40:13.361476 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.361466    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy"
Sep 08 20:40:13.361629 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.361481    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d6df25-0e7b-4cf6-b4da-4b84b98a5091" containerName="istio-proxy"
Sep 08 20:40:13.365020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.364981    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"
Sep 08 20:40:13.367516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.367494    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:40:13.370610 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.370587    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"]
Sep 08 20:40:13.430125 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.430086    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884393c3-86fb-4ae7-bca6-95f9f705e10d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-91d46f-jv5zt\" (UID: \"884393c3-86fb-4ae7-bca6-95f9f705e10d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"
Sep 08 20:40:13.531842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.531789    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884393c3-86fb-4ae7-bca6-95f9f705e10d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-91d46f-jv5zt\" (UID: \"884393c3-86fb-4ae7-bca6-95f9f705e10d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"
Sep 08 20:40:13.532713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.532679    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884393c3-86fb-4ae7-bca6-95f9f705e10d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-91d46f-jv5zt\" (UID: \"884393c3-86fb-4ae7-bca6-95f9f705e10d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"
Sep 08 20:40:13.604045 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.604001    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:40:13.604251 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.604225    2561 patch_prober.go:28] interesting pod/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 08 20:40:13.604334 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.604289    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 08 20:40:13.678487 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.678427    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"
Sep 08 20:40:13.979531 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:13.979487    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"]
Sep 08 20:40:13.980311 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:40:13.980279    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod884393c3_86fb_4ae7_bca6_95f9f705e10d.slice/crio-ec9ef803e26918918b2cdf6c30bd3e9935f926675fb4e3d4ad7ce8a9f1a1b3ef WatchSource:0}: Error finding container ec9ef803e26918918b2cdf6c30bd3e9935f926675fb4e3d4ad7ce8a9f1a1b3ef: Status 404 returned error can't find the container with id ec9ef803e26918918b2cdf6c30bd3e9935f926675fb4e3d4ad7ce8a9f1a1b3ef
Sep 08 20:40:14.396486 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:14.396400    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:40:14.396834 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:14.396815    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:40:14.604458 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:14.604424    2561 patch_prober.go:28] interesting pod/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 08 20:40:14.604627 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:14.604476    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 08 20:40:14.668348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:14.668311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt" event={"ID":"884393c3-86fb-4ae7-bca6-95f9f705e10d","Type":"ContainerStarted","Data":"ec9ef803e26918918b2cdf6c30bd3e9935f926675fb4e3d4ad7ce8a9f1a1b3ef"}
Sep 08 20:40:15.519058 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.519022    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"]
Sep 08 20:40:15.523754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.523709    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"
Sep 08 20:40:15.529658 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.529616    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"]
Sep 08 20:40:15.603940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.603634    2561 patch_prober.go:28] interesting pod/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 08 20:40:15.603940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.603690    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 08 20:40:15.676819 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.676756    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae60dae-0800-43a1-a7bb-ac4773ff1832-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr\" (UID: \"6ae60dae-0800-43a1-a7bb-ac4773ff1832\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"
Sep 08 20:40:15.777525 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.777428    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae60dae-0800-43a1-a7bb-ac4773ff1832-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr\" (UID: \"6ae60dae-0800-43a1-a7bb-ac4773ff1832\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"
Sep 08 20:40:15.778430 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.778372    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae60dae-0800-43a1-a7bb-ac4773ff1832-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr\" (UID: \"6ae60dae-0800-43a1-a7bb-ac4773ff1832\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"
Sep 08 20:40:15.865253 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:15.865185    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"
Sep 08 20:40:16.179310 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:16.179283    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"]
Sep 08 20:40:16.183170 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:40:16.182962    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae60dae_0800_43a1_a7bb_ac4773ff1832.slice/crio-ef307f498507d88287a136009087e446fe450574afed7fadd2264735d4f3cc42 WatchSource:0}: Error finding container ef307f498507d88287a136009087e446fe450574afed7fadd2264735d4f3cc42: Status 404 returned error can't find the container with id ef307f498507d88287a136009087e446fe450574afed7fadd2264735d4f3cc42
Sep 08 20:40:16.604747 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:16.604665    2561 patch_prober.go:28] interesting pod/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 08 20:40:16.605164 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:16.604738    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 08 20:40:16.685370 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:16.685329    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr" event={"ID":"6ae60dae-0800-43a1-a7bb-ac4773ff1832","Type":"ContainerStarted","Data":"ef307f498507d88287a136009087e446fe450574afed7fadd2264735d4f3cc42"}
Sep 08 20:40:17.604365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:17.604318    2561 patch_prober.go:28] interesting pod/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 08 20:40:17.604576 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:17.604401    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 08 20:40:18.607972 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:18.607940    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:40:18.608431 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:18.608309    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:40:18.609172 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:18.609152    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:40:18.700554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:18.700461    2561 generic.go:356] "Generic (PLEG): container finished" podID="884393c3-86fb-4ae7-bca6-95f9f705e10d" containerID="da99f61f706b46a0eb6876eb4c5104af395b65f8329f32080c0addb1e6627d6a" exitCode=0
Sep 08 20:40:18.700715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:18.700548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt" event={"ID":"884393c3-86fb-4ae7-bca6-95f9f705e10d","Type":"ContainerDied","Data":"da99f61f706b46a0eb6876eb4c5104af395b65f8329f32080c0addb1e6627d6a"}
Sep 08 20:40:18.702253 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:18.702226    2561 generic.go:356] "Generic (PLEG): container finished" podID="6ae60dae-0800-43a1-a7bb-ac4773ff1832" containerID="2590248b4b4fa280256ccdd2b124eb53cee613d57a0ec6ddba773fda616bbf83" exitCode=0
Sep 08 20:40:18.702373 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:18.702311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr" event={"ID":"6ae60dae-0800-43a1-a7bb-ac4773ff1832","Type":"ContainerDied","Data":"2590248b4b4fa280256ccdd2b124eb53cee613d57a0ec6ddba773fda616bbf83"}
Sep 08 20:40:19.860973 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:19.860947    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"
Sep 08 20:40:19.891690 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:19.891659    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"
Sep 08 20:40:20.021652 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.021538    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae60dae-0800-43a1-a7bb-ac4773ff1832-maas-api-service-ca\") pod \"6ae60dae-0800-43a1-a7bb-ac4773ff1832\" (UID: \"6ae60dae-0800-43a1-a7bb-ac4773ff1832\") "
Sep 08 20:40:20.021844 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.021669    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884393c3-86fb-4ae7-bca6-95f9f705e10d-maas-api-service-ca\") pod \"884393c3-86fb-4ae7-bca6-95f9f705e10d\" (UID: \"884393c3-86fb-4ae7-bca6-95f9f705e10d\") "
Sep 08 20:40:20.022018 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.021994    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae60dae-0800-43a1-a7bb-ac4773ff1832-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ae60dae-0800-43a1-a7bb-ac4773ff1832" (UID: "6ae60dae-0800-43a1-a7bb-ac4773ff1832"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:40:20.022119 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.022099    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/884393c3-86fb-4ae7-bca6-95f9f705e10d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "884393c3-86fb-4ae7-bca6-95f9f705e10d" (UID: "884393c3-86fb-4ae7-bca6-95f9f705e10d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:40:20.123162 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.123119    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884393c3-86fb-4ae7-bca6-95f9f705e10d-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:20.123162 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.123154    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae60dae-0800-43a1-a7bb-ac4773ff1832-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:20.722826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.716799    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt" event={"ID":"884393c3-86fb-4ae7-bca6-95f9f705e10d","Type":"ContainerDied","Data":"ec9ef803e26918918b2cdf6c30bd3e9935f926675fb4e3d4ad7ce8a9f1a1b3ef"}
Sep 08 20:40:20.722826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.716932    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec9ef803e26918918b2cdf6c30bd3e9935f926675fb4e3d4ad7ce8a9f1a1b3ef"
Sep 08 20:40:20.722826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.717014    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"
Sep 08 20:40:20.725275 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.725246    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr" event={"ID":"6ae60dae-0800-43a1-a7bb-ac4773ff1832","Type":"ContainerDied","Data":"ef307f498507d88287a136009087e446fe450574afed7fadd2264735d4f3cc42"}
Sep 08 20:40:20.725436 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.725277    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"
Sep 08 20:40:20.725436 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:20.725289    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef307f498507d88287a136009087e446fe450574afed7fadd2264735d4f3cc42"
Sep 08 20:40:21.665230 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:21.665200    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"]
Sep 08 20:40:21.668094 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:21.668068    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b2d463b5-q6mcr"]
Sep 08 20:40:22.364501 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:22.364464    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae60dae-0800-43a1-a7bb-ac4773ff1832" path="/var/lib/kubelet/pods/6ae60dae-0800-43a1-a7bb-ac4773ff1832/volumes"
Sep 08 20:40:22.746220 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:22.746180    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"]
Sep 08 20:40:22.746712 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:22.746419    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy" containerID="cri-o://faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d" gracePeriod=30
Sep 08 20:40:23.453337 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.453303    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"]
Sep 08 20:40:23.454012 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.453985    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae60dae-0800-43a1-a7bb-ac4773ff1832" containerName="revoke-keys"
Sep 08 20:40:23.454012 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.454013    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae60dae-0800-43a1-a7bb-ac4773ff1832" containerName="revoke-keys"
Sep 08 20:40:23.454260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.454032    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="884393c3-86fb-4ae7-bca6-95f9f705e10d" containerName="revoke-keys"
Sep 08 20:40:23.454260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.454040    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="884393c3-86fb-4ae7-bca6-95f9f705e10d" containerName="revoke-keys"
Sep 08 20:40:23.454260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.454186    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae60dae-0800-43a1-a7bb-ac4773ff1832" containerName="revoke-keys"
Sep 08 20:40:23.454260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.454206    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="884393c3-86fb-4ae7-bca6-95f9f705e10d" containerName="revoke-keys"
Sep 08 20:40:23.458407 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.458381    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.461230 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.461207    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ad269717-openshift-default-dockercfg-6q82g\""
Sep 08 20:40:23.472235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.472205    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"]
Sep 08 20:40:23.561822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.561784    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-data\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.562019 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.561869    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-envoy\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.562019 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.561976    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istiod-ca-cert\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.562118 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.562046    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-socket\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.562118 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.562068    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-certs\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.562118 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.562087    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-token\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.562223 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.562193    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-credential-socket\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.562258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.562241    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-podinfo\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.562297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.562263    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsvdb\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-kube-api-access-xsvdb\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663319    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-credential-socket\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663387    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-podinfo\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663414    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsvdb\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-kube-api-access-xsvdb\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663452    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-data\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663519    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-envoy\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istiod-ca-cert\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663863 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663643    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-socket\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663863 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663672    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-certs\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.663863 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.663696    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-token\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.664233 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.664201    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-data\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.664233 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.664225    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-credential-socket\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.665087 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.664641    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-socket\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.665087 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.664691    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-certs\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.665290 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.665106    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istiod-ca-cert\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.666928 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.666886    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-envoy\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.667387 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.667366    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-podinfo\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.677152 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.677078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-token\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.678101 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.678051    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsvdb\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-kube-api-access-xsvdb\") pod \"e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") " pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:23.774141 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:23.774020    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:24.072852 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.072822    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"]
Sep 08 20:40:24.076089 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:40:24.076049    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02087b8b_ac14_4634_b5c8_2f86b6b37c6f.slice/crio-df63f03654924887c508f9acb189f1310bafc0c1b89543ae33f3244739775e83 WatchSource:0}: Error finding container df63f03654924887c508f9acb189f1310bafc0c1b89543ae33f3244739775e83: Status 404 returned error can't find the container with id df63f03654924887c508f9acb189f1310bafc0c1b89543ae33f3244739775e83
Sep 08 20:40:24.078761 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.078711    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:24.078861 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.078826    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:24.078928 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.078867    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:24.218914 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.216414    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"]
Sep 08 20:40:24.218914 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.216640    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc" podUID="72690696-02a1-4e86-ae68-4514dff67eba" containerName="payload-processing" containerID="cri-o://f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16" gracePeriod=30
Sep 08 20:40:24.225919 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.225128    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"]
Sep 08 20:40:24.225919 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.225533    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn" podUID="b126292e-2a41-4199-9990-aade94b24353" containerName="payload-pre-processing" containerID="cri-o://481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02" gracePeriod=30
Sep 08 20:40:24.410786 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.410736    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"]
Sep 08 20:40:24.416247 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.416200    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-91d46f-jv5zt"]
Sep 08 20:40:24.534776 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.534294    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:40:24.548215 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.548178    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:40:24.579210 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.579174    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b5vf\" (UniqueName: \"kubernetes.io/projected/b126292e-2a41-4199-9990-aade94b24353-kube-api-access-6b5vf\") pod \"b126292e-2a41-4199-9990-aade94b24353\" (UID: \"b126292e-2a41-4199-9990-aade94b24353\") "
Sep 08 20:40:24.579391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.579261    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b126292e-2a41-4199-9990-aade94b24353-plugins-config-volume\") pod \"b126292e-2a41-4199-9990-aade94b24353\" (UID: \"b126292e-2a41-4199-9990-aade94b24353\") "
Sep 08 20:40:24.579391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.579355    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72690696-02a1-4e86-ae68-4514dff67eba-plugins-config-volume\") pod \"72690696-02a1-4e86-ae68-4514dff67eba\" (UID: \"72690696-02a1-4e86-ae68-4514dff67eba\") "
Sep 08 20:40:24.579536 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.579413    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkdt9\" (UniqueName: \"kubernetes.io/projected/72690696-02a1-4e86-ae68-4514dff67eba-kube-api-access-bkdt9\") pod \"72690696-02a1-4e86-ae68-4514dff67eba\" (UID: \"72690696-02a1-4e86-ae68-4514dff67eba\") "
Sep 08 20:40:24.579687 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.579663    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b126292e-2a41-4199-9990-aade94b24353-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b126292e-2a41-4199-9990-aade94b24353" (UID: "b126292e-2a41-4199-9990-aade94b24353"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:40:24.579769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.579728    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72690696-02a1-4e86-ae68-4514dff67eba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72690696-02a1-4e86-ae68-4514dff67eba" (UID: "72690696-02a1-4e86-ae68-4514dff67eba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:40:24.581608 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.581581    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b126292e-2a41-4199-9990-aade94b24353-kube-api-access-6b5vf" (OuterVolumeSpecName: "kube-api-access-6b5vf") pod "b126292e-2a41-4199-9990-aade94b24353" (UID: "b126292e-2a41-4199-9990-aade94b24353"). InnerVolumeSpecName "kube-api-access-6b5vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:40:24.581719 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.581691    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72690696-02a1-4e86-ae68-4514dff67eba-kube-api-access-bkdt9" (OuterVolumeSpecName: "kube-api-access-bkdt9") pod "72690696-02a1-4e86-ae68-4514dff67eba" (UID: "72690696-02a1-4e86-ae68-4514dff67eba"). InnerVolumeSpecName "kube-api-access-bkdt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:40:24.680694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.680661    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b126292e-2a41-4199-9990-aade94b24353-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:24.680694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.680691    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72690696-02a1-4e86-ae68-4514dff67eba-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:24.680916 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.680702    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkdt9\" (UniqueName: \"kubernetes.io/projected/72690696-02a1-4e86-ae68-4514dff67eba-kube-api-access-bkdt9\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:24.680916 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.680716    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b5vf\" (UniqueName: \"kubernetes.io/projected/b126292e-2a41-4199-9990-aade94b24353-kube-api-access-6b5vf\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:24.764268 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.764230    2561 generic.go:356] "Generic (PLEG): container finished" podID="72690696-02a1-4e86-ae68-4514dff67eba" containerID="f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16" exitCode=0
Sep 08 20:40:24.764445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.764276    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"
Sep 08 20:40:24.764445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.764316    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc" event={"ID":"72690696-02a1-4e86-ae68-4514dff67eba","Type":"ContainerDied","Data":"f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16"}
Sep 08 20:40:24.764445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.764374    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc" event={"ID":"72690696-02a1-4e86-ae68-4514dff67eba","Type":"ContainerDied","Data":"0824a721ac37e83154315b7e71e92792c9340187eab48ea37e9012110ceb71da"}
Sep 08 20:40:24.764445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.764392    2561 scope.go:117] "RemoveContainer" containerID="f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16"
Sep 08 20:40:24.766292 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.766260    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" event={"ID":"02087b8b-ac14-4634-b5c8-2f86b6b37c6f","Type":"ContainerStarted","Data":"b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698"}
Sep 08 20:40:24.766418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.766319    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" event={"ID":"02087b8b-ac14-4634-b5c8-2f86b6b37c6f","Type":"ContainerStarted","Data":"df63f03654924887c508f9acb189f1310bafc0c1b89543ae33f3244739775e83"}
Sep 08 20:40:24.767923 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.767875    2561 generic.go:356] "Generic (PLEG): container finished" podID="b126292e-2a41-4199-9990-aade94b24353" containerID="481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02" exitCode=0
Sep 08 20:40:24.768124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.767939    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"
Sep 08 20:40:24.768124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.767949    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn" event={"ID":"b126292e-2a41-4199-9990-aade94b24353","Type":"ContainerDied","Data":"481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02"}
Sep 08 20:40:24.768124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.768030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn" event={"ID":"b126292e-2a41-4199-9990-aade94b24353","Type":"ContainerDied","Data":"c183bba850e2d98762fbea1ed202bc444ae199117149799e26ebb789ee27ccd4"}
Sep 08 20:40:24.774114 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.774095    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:24.775729 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.775710    2561 scope.go:117] "RemoveContainer" containerID="f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16"
Sep 08 20:40:24.776111 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:40:24.776092    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16\": container with ID starting with f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16 not found: ID does not exist" containerID="f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16"
Sep 08 20:40:24.776204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.776120    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16"} err="failed to get container status \"f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16\": rpc error: code = NotFound desc = could not find container \"f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16\": container with ID starting with f89099dcc37e8f98c66d914f93952be3c771de9345646b59fca72707c8e38e16 not found: ID does not exist"
Sep 08 20:40:24.776204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.776140    2561 scope.go:117] "RemoveContainer" containerID="481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02"
Sep 08 20:40:24.786365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.786189    2561 scope.go:117] "RemoveContainer" containerID="481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02"
Sep 08 20:40:24.786532 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:40:24.786492    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02\": container with ID starting with 481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02 not found: ID does not exist" containerID="481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02"
Sep 08 20:40:24.786532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.786517    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02"} err="failed to get container status \"481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02\": rpc error: code = NotFound desc = could not find container \"481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02\": container with ID starting with 481067cd6b83d6b1c7028846e744956171d75b6b12f43653a712d7e0df284a02 not found: ID does not exist"
Sep 08 20:40:24.794768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.794712    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" podStartSLOduration=1.794695864 podStartE2EDuration="1.794695864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:40:24.789838253 +0000 UTC m=+910.955008829" watchObservedRunningTime="2026-09-08 20:40:24.794695864 +0000 UTC m=+910.959866442"
Sep 08 20:40:24.809105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.809070    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"]
Sep 08 20:40:24.812065 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.812041    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-91d46f-6bc4bcb5cc-msndc"]
Sep 08 20:40:24.822710 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.822680    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"]
Sep 08 20:40:24.828926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:24.828885    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-91d46f-5c7d6bc474-6k4bn"]
Sep 08 20:40:25.780528 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:25.780498    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:26.364298 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:26.364264    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72690696-02a1-4e86-ae68-4514dff67eba" path="/var/lib/kubelet/pods/72690696-02a1-4e86-ae68-4514dff67eba/volumes"
Sep 08 20:40:26.364616 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:26.364602    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="884393c3-86fb-4ae7-bca6-95f9f705e10d" path="/var/lib/kubelet/pods/884393c3-86fb-4ae7-bca6-95f9f705e10d/volumes"
Sep 08 20:40:26.364963 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:26.364949    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b126292e-2a41-4199-9990-aade94b24353" path="/var/lib/kubelet/pods/b126292e-2a41-4199-9990-aade94b24353/volumes"
Sep 08 20:40:26.784393 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:26.784360    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:26.785575 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:26.785554    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:40:28.021486 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.021414    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:40:28.117882 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.117838    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-data\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.117887    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-socket\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.117940    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-certs\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.117967    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-podinfo\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.117995    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-envoy\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118036    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-token\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118087    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-credential-socket\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118409 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118115    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istiod-ca-cert\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118409 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118170    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf5p5\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-kube-api-access-vf5p5\") pod \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\" (UID: \"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9\") "
Sep 08 20:40:28.118409 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118196    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-data" (OuterVolumeSpecName: "istio-data") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:28.118409 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118287    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:28.118409 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118321    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:28.118678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118477    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:28.118678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118626    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.118678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118650    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.118678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118667    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.118843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118683    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.118953 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.118884    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:40:28.120624 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.120600    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:40:28.120746 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.120691    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-token" (OuterVolumeSpecName: "istio-token") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:40:28.120803 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.120748    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:28.120855 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.120835    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-kube-api-access-vf5p5" (OuterVolumeSpecName: "kube-api-access-vf5p5") pod "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" (UID: "c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9"). InnerVolumeSpecName "kube-api-access-vf5p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:40:28.219851 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.219802    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.219851 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.219839    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.219851 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.219849    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.219851 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.219857    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.220168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.219867    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf5p5\" (UniqueName: \"kubernetes.io/projected/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9-kube-api-access-vf5p5\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:28.798725 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.798691    2561 generic.go:356] "Generic (PLEG): container finished" podID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerID="faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d" exitCode=0
Sep 08 20:40:28.798919 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.798740    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"
Sep 08 20:40:28.798919 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.798777    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" event={"ID":"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9","Type":"ContainerDied","Data":"faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d"}
Sep 08 20:40:28.798919 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.798836    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h" event={"ID":"c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9","Type":"ContainerDied","Data":"84412bb8471a1b1c3f83c9b96c84402cc6834e4c2036d10afcdaf1c579763ca1"}
Sep 08 20:40:28.798919 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.798858    2561 scope.go:117] "RemoveContainer" containerID="faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d"
Sep 08 20:40:28.808788 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.808767    2561 scope.go:117] "RemoveContainer" containerID="cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415"
Sep 08 20:40:28.818531 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.818500    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"]
Sep 08 20:40:28.818754 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.818724    2561 scope.go:117] "RemoveContainer" containerID="faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d"
Sep 08 20:40:28.819105 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:40:28.819084    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d\": container with ID starting with faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d not found: ID does not exist" containerID="faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d"
Sep 08 20:40:28.819197 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.819111    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d"} err="failed to get container status \"faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d\": rpc error: code = NotFound desc = could not find container \"faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d\": container with ID starting with faf032aa1820be7a945ef19b6f782abafb6d959b58fc89879f5ce1a455b1fe4d not found: ID does not exist"
Sep 08 20:40:28.819197 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.819128    2561 scope.go:117] "RemoveContainer" containerID="cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415"
Sep 08 20:40:28.819412 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:40:28.819397    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415\": container with ID starting with cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415 not found: ID does not exist" containerID="cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415"
Sep 08 20:40:28.819453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.819417    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415"} err="failed to get container status \"cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415\": rpc error: code = NotFound desc = could not find container \"cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415\": container with ID starting with cb44061a1adb3b89b06de93611bb71cd3fef5d5e26eb64b9bc4ac8ce84d91415 not found: ID does not exist"
Sep 08 20:40:28.824948 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:28.824914    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b2d463b5-openshift-default-8d65cb9dd-frt7h"]
Sep 08 20:40:30.364511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:30.364478    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" path="/var/lib/kubelet/pods/c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9/volumes"
Sep 08 20:40:38.974865 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.974825    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"]
Sep 08 20:40:38.975339 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975318    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72690696-02a1-4e86-ae68-4514dff67eba" containerName="payload-processing"
Sep 08 20:40:38.975413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975342    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="72690696-02a1-4e86-ae68-4514dff67eba" containerName="payload-processing"
Sep 08 20:40:38.975413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975357    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b126292e-2a41-4199-9990-aade94b24353" containerName="payload-pre-processing"
Sep 08 20:40:38.975413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975370    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b126292e-2a41-4199-9990-aade94b24353" containerName="payload-pre-processing"
Sep 08 20:40:38.975413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975380    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy"
Sep 08 20:40:38.975413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975392    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy"
Sep 08 20:40:38.975628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975554    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b126292e-2a41-4199-9990-aade94b24353" containerName="payload-pre-processing"
Sep 08 20:40:38.975628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975568    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy"
Sep 08 20:40:38.975628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975577    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy"
Sep 08 20:40:38.975628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.975586    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="72690696-02a1-4e86-ae68-4514dff67eba" containerName="payload-processing"
Sep 08 20:40:38.980540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.980518    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:38.983013 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.982990    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-b7a16698-kserve-self-signed-certs\""
Sep 08 20:40:38.993162 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:38.993129    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"]
Sep 08 20:40:39.024624 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.024586    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-model-cache\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.024797 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.024645    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-dshm\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.024797 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.024680    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6q2b\" (UniqueName: \"kubernetes.io/projected/0465eae3-5c07-492e-977c-ef098e02fd37-kube-api-access-l6q2b\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.024797 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.024738    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-tmp-dir\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.024797 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.024794    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-home\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.024995 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.024823    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0465eae3-5c07-492e-977c-ef098e02fd37-tls-certs\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126190 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126147    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-model-cache\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126190 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126208    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-dshm\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126471 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126236    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6q2b\" (UniqueName: \"kubernetes.io/projected/0465eae3-5c07-492e-977c-ef098e02fd37-kube-api-access-l6q2b\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126471 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126292    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-tmp-dir\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126471 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126349    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-home\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126471 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126376    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0465eae3-5c07-492e-977c-ef098e02fd37-tls-certs\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126774 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-model-cache\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126846 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126787    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-tmp-dir\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.126846 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.126815    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-home\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.128561 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.128536    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-dshm\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.128803 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.128785    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0465eae3-5c07-492e-977c-ef098e02fd37-tls-certs\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.136276 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.136251    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6q2b\" (UniqueName: \"kubernetes.io/projected/0465eae3-5c07-492e-977c-ef098e02fd37-kube-api-access-l6q2b\") pod \"e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") " pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.299598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.299514    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:39.587674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.587644    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"]
Sep 08 20:40:39.588076 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:40:39.588038    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0465eae3_5c07_492e_977c_ef098e02fd37.slice/crio-ca6e3f046432989c7daef1e3dab3391baaa2bedb4cba1e5d190f86fa3321fa4e WatchSource:0}: Error finding container ca6e3f046432989c7daef1e3dab3391baaa2bedb4cba1e5d190f86fa3321fa4e: Status 404 returned error can't find the container with id ca6e3f046432989c7daef1e3dab3391baaa2bedb4cba1e5d190f86fa3321fa4e
Sep 08 20:40:39.872934 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.872812    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt" event={"ID":"0465eae3-5c07-492e-977c-ef098e02fd37","Type":"ContainerStarted","Data":"d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba"}
Sep 08 20:40:39.872934 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.872854    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt" event={"ID":"0465eae3-5c07-492e-977c-ef098e02fd37","Type":"ContainerStarted","Data":"ca6e3f046432989c7daef1e3dab3391baaa2bedb4cba1e5d190f86fa3321fa4e"}
Sep 08 20:40:39.893063 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:39.893009    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt" podStartSLOduration=1.8929924580000002 podStartE2EDuration="1.892992458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:40:39.8902353 +0000 UTC m=+926.055405877" watchObservedRunningTime="2026-09-08 20:40:39.892992458 +0000 UTC m=+926.058163034"
Sep 08 20:40:49.300375 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:49.300326    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:49.300936 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:49.300401    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:49.313545 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:49.313511    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:49.970169 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:49.970136    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:40:50.963733 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:50.963700    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"]
Sep 08 20:40:50.964443 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:50.964419    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy"
Sep 08 20:40:50.964555 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:50.964447    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ca3a29-7cd6-4530-a5e9-3a70fb02aea9" containerName="istio-proxy"
Sep 08 20:40:50.969379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:50.969353    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"
Sep 08 20:40:50.972005 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:50.971979    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:40:50.973293 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:50.973267    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"]
Sep 08 20:40:51.053921 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:51.053191    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ae0274-1703-4b93-8108-18124fa3ddf9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ad269717-x78bw\" (UID: \"19ae0274-1703-4b93-8108-18124fa3ddf9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"
Sep 08 20:40:51.155283 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:51.154275    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ae0274-1703-4b93-8108-18124fa3ddf9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ad269717-x78bw\" (UID: \"19ae0274-1703-4b93-8108-18124fa3ddf9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"
Sep 08 20:40:51.155283 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:51.155221    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ae0274-1703-4b93-8108-18124fa3ddf9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ad269717-x78bw\" (UID: \"19ae0274-1703-4b93-8108-18124fa3ddf9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"
Sep 08 20:40:51.284400 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:51.284316    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"
Sep 08 20:40:51.584525 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:51.584489    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"]
Sep 08 20:40:51.585556 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:40:51.585525    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ae0274_1703_4b93_8108_18124fa3ddf9.slice/crio-e518b1a9a8e250afe5b09db84f46ddf8a27ab9421650425fd4a015fdc4b74cc4 WatchSource:0}: Error finding container e518b1a9a8e250afe5b09db84f46ddf8a27ab9421650425fd4a015fdc4b74cc4: Status 404 returned error can't find the container with id e518b1a9a8e250afe5b09db84f46ddf8a27ab9421650425fd4a015fdc4b74cc4
Sep 08 20:40:51.973798 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:51.973754    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw" event={"ID":"19ae0274-1703-4b93-8108-18124fa3ddf9","Type":"ContainerStarted","Data":"e518b1a9a8e250afe5b09db84f46ddf8a27ab9421650425fd4a015fdc4b74cc4"}
Sep 08 20:40:52.981874 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:52.981770    2561 generic.go:356] "Generic (PLEG): container finished" podID="19ae0274-1703-4b93-8108-18124fa3ddf9" containerID="9e729f9c57559e9ec426795e11da57c5704ad8122a902aeddb0f3ce625f70085" exitCode=0
Sep 08 20:40:52.981874 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:52.981861    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw" event={"ID":"19ae0274-1703-4b93-8108-18124fa3ddf9","Type":"ContainerDied","Data":"9e729f9c57559e9ec426795e11da57c5704ad8122a902aeddb0f3ce625f70085"}
Sep 08 20:40:54.181917 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:54.181777    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"
Sep 08 20:40:54.291127 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:54.291035    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ae0274-1703-4b93-8108-18124fa3ddf9-maas-api-service-ca\") pod \"19ae0274-1703-4b93-8108-18124fa3ddf9\" (UID: \"19ae0274-1703-4b93-8108-18124fa3ddf9\") "
Sep 08 20:40:54.291365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:54.291339    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19ae0274-1703-4b93-8108-18124fa3ddf9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "19ae0274-1703-4b93-8108-18124fa3ddf9" (UID: "19ae0274-1703-4b93-8108-18124fa3ddf9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:40:54.291453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:54.291441    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19ae0274-1703-4b93-8108-18124fa3ddf9-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:40:54.996389 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:54.996360    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"
Sep 08 20:40:54.996598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:54.996412    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw" event={"ID":"19ae0274-1703-4b93-8108-18124fa3ddf9","Type":"ContainerDied","Data":"e518b1a9a8e250afe5b09db84f46ddf8a27ab9421650425fd4a015fdc4b74cc4"}
Sep 08 20:40:54.996598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:54.996475    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e518b1a9a8e250afe5b09db84f46ddf8a27ab9421650425fd4a015fdc4b74cc4"
Sep 08 20:40:57.714703 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:57.714660    2561 patch_prober.go:28] interesting pod/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": read tcp 10.132.0.2:35142->10.132.0.72:15021: read: connection reset by peer" start-of-body=
Sep 08 20:40:57.715095 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:57.714720    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": read tcp 10.132.0.2:35142->10.132.0.72:15021: read: connection reset by peer"
Sep 08 20:40:58.007830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.007727    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"]
Sep 08 20:40:58.010552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.010520    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ad269717-x78bw"]
Sep 08 20:40:58.019205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.019176    2561 generic.go:356] "Generic (PLEG): container finished" podID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerID="b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698" exitCode=0
Sep 08 20:40:58.019359 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.019249    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" event={"ID":"02087b8b-ac14-4634-b5c8-2f86b6b37c6f","Type":"ContainerDied","Data":"b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698"}
Sep 08 20:40:58.019672 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.019658    2561 scope.go:117] "RemoveContainer" containerID="b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698"
Sep 08 20:40:58.020782 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.020756    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:58.020875 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.020838    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:58.020986 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.020926    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:40:58.151588 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.151549    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"]
Sep 08 20:40:58.365336 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:58.365255    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19ae0274-1703-4b93-8108-18124fa3ddf9" path="/var/lib/kubelet/pods/19ae0274-1703-4b93-8108-18124fa3ddf9/volumes"
Sep 08 20:40:59.030256 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:59.030222    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" event={"ID":"02087b8b-ac14-4634-b5c8-2f86b6b37c6f","Type":"ContainerStarted","Data":"90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7"}
Sep 08 20:40:59.030728 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:40:59.030309    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerName="istio-proxy" containerID="cri-o://90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7" gracePeriod=30
Sep 08 20:41:04.300883 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.300805    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:41:04.395342 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395303    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-socket\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395356    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-podinfo\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395419    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-certs\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395451    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-data\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395480    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-envoy\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395531    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-token\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395562    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-credential-socket\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395623    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istiod-ca-cert\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395648    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:04.395745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395675    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsvdb\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-kube-api-access-xsvdb\") pod \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\" (UID: \"02087b8b-ac14-4634-b5c8-2f86b6b37c6f\") "
Sep 08 20:41:04.395995 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395789    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:04.395995 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.395804    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-data" (OuterVolumeSpecName: "istio-data") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:04.396077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.396034    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:41:04.396141 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.396109    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:04.396141 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.396129    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:04.396244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.396144    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:04.396244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.396202    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:04.397867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.397836    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:41:04.397867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.397838    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-kube-api-access-xsvdb" (OuterVolumeSpecName: "kube-api-access-xsvdb") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "kube-api-access-xsvdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:41:04.398070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.397948    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-token" (OuterVolumeSpecName: "istio-token") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:41:04.398070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.398027    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "02087b8b-ac14-4634-b5c8-2f86b6b37c6f" (UID: "02087b8b-ac14-4634-b5c8-2f86b6b37c6f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:04.497553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.497504    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:04.497553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.497545    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsvdb\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-kube-api-access-xsvdb\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:04.497553 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.497558    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:04.497800 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.497567    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:04.497800 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.497575    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:04.497800 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:04.497583    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02087b8b-ac14-4634-b5c8-2f86b6b37c6f-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:05.075877 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.075838    2561 generic.go:356] "Generic (PLEG): container finished" podID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerID="90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7" exitCode=0
Sep 08 20:41:05.076086 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.075873    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" event={"ID":"02087b8b-ac14-4634-b5c8-2f86b6b37c6f","Type":"ContainerDied","Data":"90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7"}
Sep 08 20:41:05.076086 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.075919    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"
Sep 08 20:41:05.076086 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.075966    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g" event={"ID":"02087b8b-ac14-4634-b5c8-2f86b6b37c6f","Type":"ContainerDied","Data":"df63f03654924887c508f9acb189f1310bafc0c1b89543ae33f3244739775e83"}
Sep 08 20:41:05.076086 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.075991    2561 scope.go:117] "RemoveContainer" containerID="90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7"
Sep 08 20:41:05.086814 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.086790    2561 scope.go:117] "RemoveContainer" containerID="b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698"
Sep 08 20:41:05.096628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.096605    2561 scope.go:117] "RemoveContainer" containerID="90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7"
Sep 08 20:41:05.097011 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:41:05.096993    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7\": container with ID starting with 90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7 not found: ID does not exist" containerID="90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7"
Sep 08 20:41:05.097072 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.097020    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7"} err="failed to get container status \"90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7\": rpc error: code = NotFound desc = could not find container \"90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7\": container with ID starting with 90819e0d07f3cfca6036b5a7b4d2eb9b8feda0f05aeb1a0a46a9c9d8b1fee0e7 not found: ID does not exist"
Sep 08 20:41:05.097072 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.097039    2561 scope.go:117] "RemoveContainer" containerID="b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698"
Sep 08 20:41:05.097330 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:41:05.097310    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698\": container with ID starting with b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698 not found: ID does not exist" containerID="b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698"
Sep 08 20:41:05.097389 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.097336    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698"} err="failed to get container status \"b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698\": rpc error: code = NotFound desc = could not find container \"b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698\": container with ID starting with b8ae803087cbe872a245e7908c7809a6e26e2ffa62864177e944e2de3495e698 not found: ID does not exist"
Sep 08 20:41:05.098742 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.098717    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"]
Sep 08 20:41:05.102949 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:05.102916    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ad269717-openshift-default-779c4c5d56-bv75g"]
Sep 08 20:41:06.372516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:06.372475    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" path="/var/lib/kubelet/pods/02087b8b-ac14-4634-b5c8-2f86b6b37c6f/volumes"
Sep 08 20:41:11.315765 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:11.315727    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"]
Sep 08 20:41:14.557951 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.557913    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"]
Sep 08 20:41:14.558377 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.558363    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerName="istio-proxy"
Sep 08 20:41:14.558426 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.558379    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerName="istio-proxy"
Sep 08 20:41:14.558426 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.558402    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19ae0274-1703-4b93-8108-18124fa3ddf9" containerName="revoke-keys"
Sep 08 20:41:14.558426 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.558408    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="19ae0274-1703-4b93-8108-18124fa3ddf9" containerName="revoke-keys"
Sep 08 20:41:14.558519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.558485    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerName="istio-proxy"
Sep 08 20:41:14.558519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.558494    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="19ae0274-1703-4b93-8108-18124fa3ddf9" containerName="revoke-keys"
Sep 08 20:41:14.558519 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.558501    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerName="istio-proxy"
Sep 08 20:41:14.563478 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.563421    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.566614 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.566590    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-b433c5-openshift-default-dockercfg-flfq9\""
Sep 08 20:41:14.575655 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.575628    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"]
Sep 08 20:41:14.603366 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603337    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-data\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.603563 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xg44\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-kube-api-access-6xg44\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.603563 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603393    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-credential-socket\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.603563 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603408    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-istio-token\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.603563 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603449    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93167841-6983-43d3-a9c4-7e54527c99a2-istiod-ca-cert\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.603563 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603491    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-socket\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.603563 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603548    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-certs\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.603846 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603623    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93167841-6983-43d3-a9c4-7e54527c99a2-istio-podinfo\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.603846 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.603692    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-envoy\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704235    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-socket\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704291    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-certs\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704334    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93167841-6983-43d3-a9c4-7e54527c99a2-istio-podinfo\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704374    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-envoy\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704398    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-data\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704423    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xg44\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-kube-api-access-6xg44\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704728 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704448    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-credential-socket\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704728 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704471    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-istio-token\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704728 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704508    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93167841-6983-43d3-a9c4-7e54527c99a2-istiod-ca-cert\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704728 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704695    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-socket\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704977 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-certs\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704977 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704794    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-data\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.704977 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.704832    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-credential-socket\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.705208 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.705189    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93167841-6983-43d3-a9c4-7e54527c99a2-istiod-ca-cert\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.706887 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.706863    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-envoy\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.707013 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.706940    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93167841-6983-43d3-a9c4-7e54527c99a2-istio-podinfo\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.713024 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.713003    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-istio-token\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.713407 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.713386    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xg44\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-kube-api-access-6xg44\") pod \"e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") " pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:14.876777 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:14.876678    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:15.160069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:15.160039    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"]
Sep 08 20:41:15.161251 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:41:15.161226    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93167841_6983_43d3_a9c4_7e54527c99a2.slice/crio-ba9bdfb219e36db5889a6c8a3f08bf4dd492dc6193438e8c64cacc2b70401333 WatchSource:0}: Error finding container ba9bdfb219e36db5889a6c8a3f08bf4dd492dc6193438e8c64cacc2b70401333: Status 404 returned error can't find the container with id ba9bdfb219e36db5889a6c8a3f08bf4dd492dc6193438e8c64cacc2b70401333
Sep 08 20:41:15.163980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:15.163943    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:41:15.164070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:15.164015    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:41:15.164070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:15.164046    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:41:16.167222 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:16.167177    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" event={"ID":"93167841-6983-43d3-a9c4-7e54527c99a2","Type":"ContainerStarted","Data":"8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7"}
Sep 08 20:41:16.167598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:16.167263    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" event={"ID":"93167841-6983-43d3-a9c4-7e54527c99a2","Type":"ContainerStarted","Data":"ba9bdfb219e36db5889a6c8a3f08bf4dd492dc6193438e8c64cacc2b70401333"}
Sep 08 20:41:16.190013 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:16.189954    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" podStartSLOduration=2.189936688 podStartE2EDuration="2.189936688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:41:16.186200681 +0000 UTC m=+962.351371257" watchObservedRunningTime="2026-09-08 20:41:16.189936688 +0000 UTC m=+962.355107258"
Sep 08 20:41:16.877789 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:16.877741    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:16.883735 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:16.883702    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:17.173684 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:17.173657    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:17.174755 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:17.174732    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:19.928064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:19.928030    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"]
Sep 08 20:41:19.928540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:19.928523    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerName="istio-proxy"
Sep 08 20:41:19.928587 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:19.928544    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="02087b8b-ac14-4634-b5c8-2f86b6b37c6f" containerName="istio-proxy"
Sep 08 20:41:19.931838 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:19.931796    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:19.936007 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:19.935977    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-b433c5-dockercfg-zlvw7\""
Sep 08 20:41:19.936350 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:19.936332    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-b433c5\""
Sep 08 20:41:19.942567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:19.942459    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"]
Sep 08 20:41:20.063804 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:20.063762    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh9wj\" (UniqueName: \"kubernetes.io/projected/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-kube-api-access-qh9wj\") pod \"payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td\" (UID: \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:20.064097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:20.063911    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td\" (UID: \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:20.164924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:20.164856    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td\" (UID: \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:20.165134 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:20.165010    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh9wj\" (UniqueName: \"kubernetes.io/projected/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-kube-api-access-qh9wj\") pod \"payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td\" (UID: \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:20.165661 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:20.165637    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td\" (UID: \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:20.173648 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:20.173613    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh9wj\" (UniqueName: \"kubernetes.io/projected/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-kube-api-access-qh9wj\") pod \"payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td\" (UID: \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:20.247182 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:20.247094    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:20.523368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:20.523331    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"]
Sep 08 20:41:20.524614 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:41:20.524587    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5804dbd2_f64a_4af5_b359_5bdc6b1c7492.slice/crio-5f09afa581a808046772348e64da7804dfdab891e3cbdd7ecb26eabb1ecfbbfb WatchSource:0}: Error finding container 5f09afa581a808046772348e64da7804dfdab891e3cbdd7ecb26eabb1ecfbbfb: Status 404 returned error can't find the container with id 5f09afa581a808046772348e64da7804dfdab891e3cbdd7ecb26eabb1ecfbbfb
Sep 08 20:41:21.204210 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:21.204160    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td" event={"ID":"5804dbd2-f64a-4af5-b359-5bdc6b1c7492","Type":"ContainerStarted","Data":"bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6"}
Sep 08 20:41:21.204609 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:21.204259    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td" event={"ID":"5804dbd2-f64a-4af5-b359-5bdc6b1c7492","Type":"ContainerStarted","Data":"5f09afa581a808046772348e64da7804dfdab891e3cbdd7ecb26eabb1ecfbbfb"}
Sep 08 20:41:21.204609 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:21.204295    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:21.223194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:21.223141    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td" podStartSLOduration=2.223124058 podStartE2EDuration="2.223124058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:41:21.221032936 +0000 UTC m=+967.386203513" watchObservedRunningTime="2026-09-08 20:41:21.223124058 +0000 UTC m=+967.388294634"
Sep 08 20:41:26.342227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.342180    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt" podUID="0465eae3-5c07-492e-977c-ef098e02fd37" containerName="main" containerID="cri-o://d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba" gracePeriod=45
Sep 08 20:41:26.609999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.609909    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:41:26.730122 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730088    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-tmp-dir\") pod \"0465eae3-5c07-492e-977c-ef098e02fd37\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") "
Sep 08 20:41:26.730297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730181    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-model-cache\") pod \"0465eae3-5c07-492e-977c-ef098e02fd37\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") "
Sep 08 20:41:26.730297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730215    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6q2b\" (UniqueName: \"kubernetes.io/projected/0465eae3-5c07-492e-977c-ef098e02fd37-kube-api-access-l6q2b\") pod \"0465eae3-5c07-492e-977c-ef098e02fd37\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") "
Sep 08 20:41:26.730297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730239    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-home\") pod \"0465eae3-5c07-492e-977c-ef098e02fd37\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") "
Sep 08 20:41:26.730297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730279    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0465eae3-5c07-492e-977c-ef098e02fd37-tls-certs\") pod \"0465eae3-5c07-492e-977c-ef098e02fd37\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") "
Sep 08 20:41:26.730475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730359    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-dshm\") pod \"0465eae3-5c07-492e-977c-ef098e02fd37\" (UID: \"0465eae3-5c07-492e-977c-ef098e02fd37\") "
Sep 08 20:41:26.730475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730374    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0465eae3-5c07-492e-977c-ef098e02fd37" (UID: "0465eae3-5c07-492e-977c-ef098e02fd37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:26.730475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730417    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-model-cache" (OuterVolumeSpecName: "model-cache") pod "0465eae3-5c07-492e-977c-ef098e02fd37" (UID: "0465eae3-5c07-492e-977c-ef098e02fd37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:26.730644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730522    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-home" (OuterVolumeSpecName: "home") pod "0465eae3-5c07-492e-977c-ef098e02fd37" (UID: "0465eae3-5c07-492e-977c-ef098e02fd37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:26.730783 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730760    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-tmp-dir\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:26.730878 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730787    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-model-cache\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:26.730878 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.730801    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-home\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:26.733450 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.733404    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0465eae3-5c07-492e-977c-ef098e02fd37-kube-api-access-l6q2b" (OuterVolumeSpecName: "kube-api-access-l6q2b") pod "0465eae3-5c07-492e-977c-ef098e02fd37" (UID: "0465eae3-5c07-492e-977c-ef098e02fd37"). InnerVolumeSpecName "kube-api-access-l6q2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:41:26.733450 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.733406    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0465eae3-5c07-492e-977c-ef098e02fd37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0465eae3-5c07-492e-977c-ef098e02fd37" (UID: "0465eae3-5c07-492e-977c-ef098e02fd37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:41:26.733450 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.733409    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-dshm" (OuterVolumeSpecName: "dshm") pod "0465eae3-5c07-492e-977c-ef098e02fd37" (UID: "0465eae3-5c07-492e-977c-ef098e02fd37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:26.832282 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.832232    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0465eae3-5c07-492e-977c-ef098e02fd37-dshm\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:26.832282 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.832269    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6q2b\" (UniqueName: \"kubernetes.io/projected/0465eae3-5c07-492e-977c-ef098e02fd37-kube-api-access-l6q2b\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:26.832282 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:26.832281    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0465eae3-5c07-492e-977c-ef098e02fd37-tls-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:41:27.247102 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.247065    2561 generic.go:356] "Generic (PLEG): container finished" podID="0465eae3-5c07-492e-977c-ef098e02fd37" containerID="d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba" exitCode=0
Sep 08 20:41:27.247309 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.247147    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"
Sep 08 20:41:27.247309 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.247134    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt" event={"ID":"0465eae3-5c07-492e-977c-ef098e02fd37","Type":"ContainerDied","Data":"d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba"}
Sep 08 20:41:27.247309 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.247277    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt" event={"ID":"0465eae3-5c07-492e-977c-ef098e02fd37","Type":"ContainerDied","Data":"ca6e3f046432989c7daef1e3dab3391baaa2bedb4cba1e5d190f86fa3321fa4e"}
Sep 08 20:41:27.247309 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.247302    2561 scope.go:117] "RemoveContainer" containerID="d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba"
Sep 08 20:41:27.260129 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.260104    2561 scope.go:117] "RemoveContainer" containerID="d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba"
Sep 08 20:41:27.260616 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:41:27.260591    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba\": container with ID starting with d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba not found: ID does not exist" containerID="d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba"
Sep 08 20:41:27.260726 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.260625    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba"} err="failed to get container status \"d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba\": rpc error: code = NotFound desc = could not find container \"d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba\": container with ID starting with d7ae60aeb5d90498c14aea18bd6e2cd0045b8fce38221ae999a0273745ad02ba not found: ID does not exist"
Sep 08 20:41:27.275703 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.275668    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"]
Sep 08 20:41:27.278375 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:27.278351    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-b7a16698-kserve-6b4bcf4487-hl5dt"]
Sep 08 20:41:28.364428 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:28.364390    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0465eae3-5c07-492e-977c-ef098e02fd37" path="/var/lib/kubelet/pods/0465eae3-5c07-492e-977c-ef098e02fd37/volumes"
Sep 08 20:41:32.212720 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:32.212684    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:41:37.327422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:37.327313    2561 generic.go:356] "Generic (PLEG): container finished" podID="93167841-6983-43d3-a9c4-7e54527c99a2" containerID="8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7" exitCode=0
Sep 08 20:41:37.327422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:37.327390    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" event={"ID":"93167841-6983-43d3-a9c4-7e54527c99a2","Type":"ContainerDied","Data":"8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7"}
Sep 08 20:41:37.328038 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:37.327959    2561 scope.go:117] "RemoveContainer" containerID="8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7"
Sep 08 20:41:37.329094 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:37.329063    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:41:37.329187 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:37.329134    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:41:37.329187 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:37.329167    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:41:38.338744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:38.338702    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" event={"ID":"93167841-6983-43d3-a9c4-7e54527c99a2","Type":"ContainerStarted","Data":"2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358"}
Sep 08 20:41:38.877912 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:38.877851    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:38.878189 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:38.878165    2561 patch_prober.go:28] interesting pod/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 08 20:41:38.878238 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:38.878216    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 08 20:41:39.877924 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:39.877859    2561 patch_prober.go:28] interesting pod/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 08 20:41:39.878403 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:39.877948    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 08 20:41:40.877193 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:40.877126    2561 patch_prober.go:28] interesting pod/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 08 20:41:40.877193 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:40.877185    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 08 20:41:41.878082 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:41.878048    2561 patch_prober.go:28] interesting pod/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 08 20:41:41.878565 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:41.878533    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 08 20:41:42.877881 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:42.877843    2561 patch_prober.go:28] interesting pod/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 08 20:41:42.878084 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:42.877927    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 08 20:41:43.881351 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:43.881316    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:43.881849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:43.881693    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:41:43.882411 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:41:43.882392    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:42:08.187149 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.187110    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"]
Sep 08 20:42:08.187617 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.187585    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0465eae3-5c07-492e-977c-ef098e02fd37" containerName="main"
Sep 08 20:42:08.187617 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.187597    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0465eae3-5c07-492e-977c-ef098e02fd37" containerName="main"
Sep 08 20:42:08.187694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.187671    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0465eae3-5c07-492e-977c-ef098e02fd37" containerName="main"
Sep 08 20:42:08.192078 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.192057    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"
Sep 08 20:42:08.194832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.194801    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:42:08.198817 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.198787    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"]
Sep 08 20:42:08.366792 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.365946    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba70f73f-b831-468d-acca-7f0da505242c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b433c5-fctrk\" (UID: \"ba70f73f-b831-468d-acca-7f0da505242c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"
Sep 08 20:42:08.467644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.467532    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba70f73f-b831-468d-acca-7f0da505242c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b433c5-fctrk\" (UID: \"ba70f73f-b831-468d-acca-7f0da505242c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"
Sep 08 20:42:08.468219 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.468199    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba70f73f-b831-468d-acca-7f0da505242c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b433c5-fctrk\" (UID: \"ba70f73f-b831-468d-acca-7f0da505242c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"
Sep 08 20:42:08.505517 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.505478    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"
Sep 08 20:42:08.784173 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:08.784144    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"]
Sep 08 20:42:08.785205 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:42:08.785175    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba70f73f_b831_468d_acca_7f0da505242c.slice/crio-1600c6da7ac25216afa474b5edaae3135231fbb4e3283ade6f4396ad22b6e7e9 WatchSource:0}: Error finding container 1600c6da7ac25216afa474b5edaae3135231fbb4e3283ade6f4396ad22b6e7e9: Status 404 returned error can't find the container with id 1600c6da7ac25216afa474b5edaae3135231fbb4e3283ade6f4396ad22b6e7e9
Sep 08 20:42:09.563332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:09.563297    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk" event={"ID":"ba70f73f-b831-468d-acca-7f0da505242c","Type":"ContainerStarted","Data":"9259907cb64b3d2aa62bf456efcb3f235ef8f2f45fd7a7b2a243a4922a46c668"}
Sep 08 20:42:09.563769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:09.563353    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk" event={"ID":"ba70f73f-b831-468d-acca-7f0da505242c","Type":"ContainerStarted","Data":"1600c6da7ac25216afa474b5edaae3135231fbb4e3283ade6f4396ad22b6e7e9"}
Sep 08 20:42:09.579623 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:09.579567    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk" podStartSLOduration=1.579550898 podStartE2EDuration="1.579550898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:42:09.578339338 +0000 UTC m=+1015.743509914" watchObservedRunningTime="2026-09-08 20:42:09.579550898 +0000 UTC m=+1015.744721473"
Sep 08 20:42:10.573514 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:10.573472    2561 generic.go:356] "Generic (PLEG): container finished" podID="ba70f73f-b831-468d-acca-7f0da505242c" containerID="9259907cb64b3d2aa62bf456efcb3f235ef8f2f45fd7a7b2a243a4922a46c668" exitCode=0
Sep 08 20:42:10.573988 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:10.573565    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk" event={"ID":"ba70f73f-b831-468d-acca-7f0da505242c","Type":"ContainerDied","Data":"9259907cb64b3d2aa62bf456efcb3f235ef8f2f45fd7a7b2a243a4922a46c668"}
Sep 08 20:42:11.724352 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:11.724322    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"
Sep 08 20:42:11.902715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:11.902679    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba70f73f-b831-468d-acca-7f0da505242c-maas-api-service-ca\") pod \"ba70f73f-b831-468d-acca-7f0da505242c\" (UID: \"ba70f73f-b831-468d-acca-7f0da505242c\") "
Sep 08 20:42:11.903120 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:11.903090    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba70f73f-b831-468d-acca-7f0da505242c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba70f73f-b831-468d-acca-7f0da505242c" (UID: "ba70f73f-b831-468d-acca-7f0da505242c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:42:12.003954 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:12.003881    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba70f73f-b831-468d-acca-7f0da505242c-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:12.588692 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:12.588661    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"
Sep 08 20:42:12.588692 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:12.588688    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk" event={"ID":"ba70f73f-b831-468d-acca-7f0da505242c","Type":"ContainerDied","Data":"1600c6da7ac25216afa474b5edaae3135231fbb4e3283ade6f4396ad22b6e7e9"}
Sep 08 20:42:12.588934 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:12.588737    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1600c6da7ac25216afa474b5edaae3135231fbb4e3283ade6f4396ad22b6e7e9"
Sep 08 20:42:18.791132 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:18.791098    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"]
Sep 08 20:42:18.791529 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:18.791311    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td" podUID="5804dbd2-f64a-4af5-b359-5bdc6b1c7492" containerName="payload-pre-processing" containerID="cri-o://bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6" gracePeriod=30
Sep 08 20:42:19.066022 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.065989    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"]
Sep 08 20:42:19.066411 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.066392    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:42:19.073801 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.073761    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b433c5-fctrk"]
Sep 08 20:42:19.168143 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.168102    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-plugins-config-volume\") pod \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\" (UID: \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\") "
Sep 08 20:42:19.168319 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.168170    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh9wj\" (UniqueName: \"kubernetes.io/projected/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-kube-api-access-qh9wj\") pod \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\" (UID: \"5804dbd2-f64a-4af5-b359-5bdc6b1c7492\") "
Sep 08 20:42:19.168595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.168573    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5804dbd2-f64a-4af5-b359-5bdc6b1c7492" (UID: "5804dbd2-f64a-4af5-b359-5bdc6b1c7492"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:42:19.170349 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.170328    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-kube-api-access-qh9wj" (OuterVolumeSpecName: "kube-api-access-qh9wj") pod "5804dbd2-f64a-4af5-b359-5bdc6b1c7492" (UID: "5804dbd2-f64a-4af5-b359-5bdc6b1c7492"). InnerVolumeSpecName "kube-api-access-qh9wj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:42:19.269576 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.269535    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:19.269576 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.269568    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh9wj\" (UniqueName: \"kubernetes.io/projected/5804dbd2-f64a-4af5-b359-5bdc6b1c7492-kube-api-access-qh9wj\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:19.646229 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.646183    2561 generic.go:356] "Generic (PLEG): container finished" podID="5804dbd2-f64a-4af5-b359-5bdc6b1c7492" containerID="bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6" exitCode=0
Sep 08 20:42:19.646422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.646233    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"
Sep 08 20:42:19.646422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.646250    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td" event={"ID":"5804dbd2-f64a-4af5-b359-5bdc6b1c7492","Type":"ContainerDied","Data":"bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6"}
Sep 08 20:42:19.646422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.646305    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td" event={"ID":"5804dbd2-f64a-4af5-b359-5bdc6b1c7492","Type":"ContainerDied","Data":"5f09afa581a808046772348e64da7804dfdab891e3cbdd7ecb26eabb1ecfbbfb"}
Sep 08 20:42:19.646422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.646328    2561 scope.go:117] "RemoveContainer" containerID="bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6"
Sep 08 20:42:19.656598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.656578    2561 scope.go:117] "RemoveContainer" containerID="bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6"
Sep 08 20:42:19.656953 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:42:19.656933    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6\": container with ID starting with bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6 not found: ID does not exist" containerID="bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6"
Sep 08 20:42:19.657040 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.656961    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6"} err="failed to get container status \"bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6\": rpc error: code = NotFound desc = could not find container \"bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6\": container with ID starting with bf46e1780928abd398243cc0bd7efc97136ef2af1cb95ee5708fec39bffd70c6 not found: ID does not exist"
Sep 08 20:42:19.669331 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.669267    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"]
Sep 08 20:42:19.672699 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:19.672669    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b433c5-5b7d9f4f4b-652td"]
Sep 08 20:42:20.364942 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:20.364880    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5804dbd2-f64a-4af5-b359-5bdc6b1c7492" path="/var/lib/kubelet/pods/5804dbd2-f64a-4af5-b359-5bdc6b1c7492/volumes"
Sep 08 20:42:20.365334 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:20.365264    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba70f73f-b831-468d-acca-7f0da505242c" path="/var/lib/kubelet/pods/ba70f73f-b831-468d-acca-7f0da505242c/volumes"
Sep 08 20:42:25.729747 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:25.729712    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"]
Sep 08 20:42:25.730317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:25.729998    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy" containerID="cri-o://2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358" gracePeriod=30
Sep 08 20:42:26.470342 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.470306    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"]
Sep 08 20:42:26.470789 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.470775    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5804dbd2-f64a-4af5-b359-5bdc6b1c7492" containerName="payload-pre-processing"
Sep 08 20:42:26.470843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.470791    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5804dbd2-f64a-4af5-b359-5bdc6b1c7492" containerName="payload-pre-processing"
Sep 08 20:42:26.470843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.470820    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba70f73f-b831-468d-acca-7f0da505242c" containerName="revoke-keys"
Sep 08 20:42:26.470843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.470826    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba70f73f-b831-468d-acca-7f0da505242c" containerName="revoke-keys"
Sep 08 20:42:26.470965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.470890    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5804dbd2-f64a-4af5-b359-5bdc6b1c7492" containerName="payload-pre-processing"
Sep 08 20:42:26.470965 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.470935    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba70f73f-b831-468d-acca-7f0da505242c" containerName="revoke-keys"
Sep 08 20:42:26.475367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.475341    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.477935 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.477912    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-458e34-openshift-default-dockercfg-m69hq\""
Sep 08 20:42:26.486959 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.486928    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"]
Sep 08 20:42:26.542988 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.542933    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-credential-socket\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.542988 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.542988    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-data\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.543323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.543012    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-podinfo\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.543323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.543036    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-envoy\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.543323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.543061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istiod-ca-cert\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.543323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.543162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-token\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.543323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.543247    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f777g\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-kube-api-access-f777g\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.543323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.543278    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-socket\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.543524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.543327    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-certs\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.644629 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644586    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-certs\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.644822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644665    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-credential-socket\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.644822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644683    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-data\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.644822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644704    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-podinfo\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.644822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644722    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-envoy\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.644822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644743    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istiod-ca-cert\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.644822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644796    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-token\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.645188 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644860    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f777g\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-kube-api-access-f777g\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.645188 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.644930    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-socket\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.645188 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.645098    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-data\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.645188 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.645116    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-certs\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.645487 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.645269    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-credential-socket\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.645487 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.645338    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-socket\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.645577 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.645512    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istiod-ca-cert\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.647301 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.647270    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-envoy\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.647498 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.647474    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-podinfo\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.657644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.657616    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-token\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.657780 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.657754    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f777g\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-kube-api-access-f777g\") pod \"e2e-noten-458e34-openshift-default-59d55c778b-dj7j4\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") " pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:26.788827 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:26.788732    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:27.076660 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:27.076564    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"]
Sep 08 20:42:27.080745 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:42:27.080709    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d4bcdc3_03c7_4747_9f71_5bf95d8adb9c.slice/crio-bdcca2e0eaa288f1c7355478c4c415e412dd4c34aba6b7001313d210950d7236 WatchSource:0}: Error finding container bdcca2e0eaa288f1c7355478c4c415e412dd4c34aba6b7001313d210950d7236: Status 404 returned error can't find the container with id bdcca2e0eaa288f1c7355478c4c415e412dd4c34aba6b7001313d210950d7236
Sep 08 20:42:27.083683 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:27.083640    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:27.083771 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:27.083741    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:27.083811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:27.083789    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:27.712378 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:27.712335    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4" event={"ID":"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c","Type":"ContainerStarted","Data":"12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2"}
Sep 08 20:42:27.712570 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:27.712436    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4" event={"ID":"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c","Type":"ContainerStarted","Data":"bdcca2e0eaa288f1c7355478c4c415e412dd4c34aba6b7001313d210950d7236"}
Sep 08 20:42:27.734173 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:27.734117    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4" podStartSLOduration=1.734098643 podStartE2EDuration="1.734098643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:42:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:42:27.730851569 +0000 UTC m=+1033.896022145" watchObservedRunningTime="2026-09-08 20:42:27.734098643 +0000 UTC m=+1033.899269219"
Sep 08 20:42:27.789625 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:27.789582    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:28.794628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:28.794598    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:28.883999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:28.883961    2561 patch_prober.go:28] interesting pod/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 20:42:28.884166 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:28.884024    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 20:42:29.726480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:29.726448    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:29.727514 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:29.727488    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:42:31.007579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.007508    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:42:31.092665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.092628    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-data\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.092866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.092702    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-envoy\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.092866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.092761    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93167841-6983-43d3-a9c4-7e54527c99a2-istiod-ca-cert\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.092866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.092795    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-socket\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.092866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.092838    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-credential-socket\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.093128 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.092871    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-certs\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.093128 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.092996    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-data" (OuterVolumeSpecName: "istio-data") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:31.093128 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093005    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xg44\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-kube-api-access-6xg44\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.093301 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093188    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93167841-6983-43d3-a9c4-7e54527c99a2-istio-podinfo\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.093301 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093190    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:31.093301 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093219    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-istio-token\") pod \"93167841-6983-43d3-a9c4-7e54527c99a2\" (UID: \"93167841-6983-43d3-a9c4-7e54527c99a2\") "
Sep 08 20:42:31.093301 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093242    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:31.093301 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093288    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93167841-6983-43d3-a9c4-7e54527c99a2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:42:31.093623 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093598    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:31.093623 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093621    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.093785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093645    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.093785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093660    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.093785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.093672    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93167841-6983-43d3-a9c4-7e54527c99a2-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.095335 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.095311    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93167841-6983-43d3-a9c4-7e54527c99a2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:42:31.095447 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.095312    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:42:31.095510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.095441    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-kube-api-access-6xg44" (OuterVolumeSpecName: "kube-api-access-6xg44") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "kube-api-access-6xg44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:42:31.095510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.095453    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93167841-6983-43d3-a9c4-7e54527c99a2" (UID: "93167841-6983-43d3-a9c4-7e54527c99a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:31.194556 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.194523    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xg44\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-kube-api-access-6xg44\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.194556 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.194554    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93167841-6983-43d3-a9c4-7e54527c99a2-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.194556 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.194563    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93167841-6983-43d3-a9c4-7e54527c99a2-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.194783 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.194571    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.194783 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.194582    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93167841-6983-43d3-a9c4-7e54527c99a2-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:42:31.740981 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.740937    2561 generic.go:356] "Generic (PLEG): container finished" podID="93167841-6983-43d3-a9c4-7e54527c99a2" containerID="2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358" exitCode=0
Sep 08 20:42:31.741148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.741025    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"
Sep 08 20:42:31.741148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.741062    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" event={"ID":"93167841-6983-43d3-a9c4-7e54527c99a2","Type":"ContainerDied","Data":"2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358"}
Sep 08 20:42:31.741233 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.741147    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg" event={"ID":"93167841-6983-43d3-a9c4-7e54527c99a2","Type":"ContainerDied","Data":"ba9bdfb219e36db5889a6c8a3f08bf4dd492dc6193438e8c64cacc2b70401333"}
Sep 08 20:42:31.741233 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.741178    2561 scope.go:117] "RemoveContainer" containerID="2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358"
Sep 08 20:42:31.752369 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.752353    2561 scope.go:117] "RemoveContainer" containerID="8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7"
Sep 08 20:42:31.761306 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.761251    2561 scope.go:117] "RemoveContainer" containerID="2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358"
Sep 08 20:42:31.761680 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:42:31.761649    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358\": container with ID starting with 2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358 not found: ID does not exist" containerID="2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358"
Sep 08 20:42:31.761770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.761693    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358"} err="failed to get container status \"2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358\": rpc error: code = NotFound desc = could not find container \"2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358\": container with ID starting with 2254bd6d36d74907d204e9e3b7c3f0e65a8dce1d746d1ce10bdeb3d251650358 not found: ID does not exist"
Sep 08 20:42:31.761770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.761721    2561 scope.go:117] "RemoveContainer" containerID="8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7"
Sep 08 20:42:31.762154 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:42:31.762133    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7\": container with ID starting with 8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7 not found: ID does not exist" containerID="8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7"
Sep 08 20:42:31.762267 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.762160    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7"} err="failed to get container status \"8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7\": rpc error: code = NotFound desc = could not find container \"8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7\": container with ID starting with 8dae7d490ce75b1fc1b8db95093322e66f361f127a73a72b3d10c38948aae9e7 not found: ID does not exist"
Sep 08 20:42:31.765211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.765192    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"]
Sep 08 20:42:31.769958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:31.769873    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-b433c5-openshift-default-d5cb9cdfb-nvnjg"]
Sep 08 20:42:32.365330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:32.365293    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" path="/var/lib/kubelet/pods/93167841-6983-43d3-a9c4-7e54527c99a2/volumes"
Sep 08 20:42:34.032831 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.032794    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"]
Sep 08 20:42:34.033475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.033454    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy"
Sep 08 20:42:34.033475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.033476    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy"
Sep 08 20:42:34.033631 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.033622    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy"
Sep 08 20:42:34.033687 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.033641    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy"
Sep 08 20:42:34.038371 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.038347    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.041117 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.041085    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3e6b3c9b-openshift-default-dockercfg-lzggz\""
Sep 08 20:42:34.050242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.050154    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"]
Sep 08 20:42:34.121065 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-envoy\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.121279 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121104    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-podinfo\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.121279 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121129    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-token\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.121279 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121202    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-data\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.121279 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121273    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lpt6\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-kube-api-access-8lpt6\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.121480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121318    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-certs\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.121480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121361    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istiod-ca-cert\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.121480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121408    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-credential-socket\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.121480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.121445    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-socket\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.222492 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222457    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lpt6\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-kube-api-access-8lpt6\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.222705 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222508    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-certs\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.222705 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222533    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istiod-ca-cert\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.222705 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222571    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-credential-socket\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.222705 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222604    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-socket\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.222705 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222639    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-envoy\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.222705 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222691    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-podinfo\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.223080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222714    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-token\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.223080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.222770    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-data\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.223080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.223013    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-certs\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.223080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.223063    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-credential-socket\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.223220 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.223202    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-socket\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.223257 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.223226    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-data\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.223363 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.223344    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istiod-ca-cert\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.225113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.225094    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-envoy\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.225337 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.225319    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-podinfo\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.230728 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.230707    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-token\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.230840 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.230741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lpt6\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-kube-api-access-8lpt6\") pod \"e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") " pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.351604 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.351509    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:34.637603 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:42:34.637163    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddacecc0a_d25d_46c3_8e07_7e6eb89e572b.slice/crio-0658f21172ebec2dc2fbe36fdbb7982570681a8466fff9cf41d4e48483fe77d6 WatchSource:0}: Error finding container 0658f21172ebec2dc2fbe36fdbb7982570681a8466fff9cf41d4e48483fe77d6: Status 404 returned error can't find the container with id 0658f21172ebec2dc2fbe36fdbb7982570681a8466fff9cf41d4e48483fe77d6
Sep 08 20:42:34.637603 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.637295    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"]
Sep 08 20:42:34.639348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.639314    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:34.639481 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.639395    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:34.639481 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.639441    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:34.766069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.766027    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" event={"ID":"dacecc0a-d25d-46c3-8e07-7e6eb89e572b","Type":"ContainerStarted","Data":"2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14"}
Sep 08 20:42:34.766254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.766097    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" event={"ID":"dacecc0a-d25d-46c3-8e07-7e6eb89e572b","Type":"ContainerStarted","Data":"0658f21172ebec2dc2fbe36fdbb7982570681a8466fff9cf41d4e48483fe77d6"}
Sep 08 20:42:34.794969 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:34.794884    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" podStartSLOduration=1.794862263 podStartE2EDuration="1.794862263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:42:34.793029216 +0000 UTC m=+1040.958199793" watchObservedRunningTime="2026-09-08 20:42:34.794862263 +0000 UTC m=+1040.960032839"
Sep 08 20:42:35.352069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:35.352033    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:35.357556 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:35.357524    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:35.773542 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:35.773506    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:35.774449 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:35.774428    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:38.739449 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.739400    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"]
Sep 08 20:42:38.740025 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.740005    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy"
Sep 08 20:42:38.740025 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.740025    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="93167841-6983-43d3-a9c4-7e54527c99a2" containerName="istio-proxy"
Sep 08 20:42:38.743526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.743503    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:38.746155 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.746123    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-3e6b3c9b\""
Sep 08 20:42:38.746304 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.746186    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-3e6b3c9b-dockercfg-457ns\""
Sep 08 20:42:38.752742 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.752457    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"]
Sep 08 20:42:38.872307 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.872267    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltn4b\" (UniqueName: \"kubernetes.io/projected/762d711e-dc62-4d8d-9579-cd57c2195775-kube-api-access-ltn4b\") pod \"payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k\" (UID: \"762d711e-dc62-4d8d-9579-cd57c2195775\") " pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:38.872475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.872406    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/762d711e-dc62-4d8d-9579-cd57c2195775-plugins-config-volume\") pod \"payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k\" (UID: \"762d711e-dc62-4d8d-9579-cd57c2195775\") " pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:38.973319 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.973277    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltn4b\" (UniqueName: \"kubernetes.io/projected/762d711e-dc62-4d8d-9579-cd57c2195775-kube-api-access-ltn4b\") pod \"payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k\" (UID: \"762d711e-dc62-4d8d-9579-cd57c2195775\") " pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:38.973510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.973359    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/762d711e-dc62-4d8d-9579-cd57c2195775-plugins-config-volume\") pod \"payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k\" (UID: \"762d711e-dc62-4d8d-9579-cd57c2195775\") " pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:38.974092 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.974071    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/762d711e-dc62-4d8d-9579-cd57c2195775-plugins-config-volume\") pod \"payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k\" (UID: \"762d711e-dc62-4d8d-9579-cd57c2195775\") " pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:38.981921 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:38.981857    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltn4b\" (UniqueName: \"kubernetes.io/projected/762d711e-dc62-4d8d-9579-cd57c2195775-kube-api-access-ltn4b\") pod \"payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k\" (UID: \"762d711e-dc62-4d8d-9579-cd57c2195775\") " pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:39.057169 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:39.057082    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:39.338753 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:39.338727    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"]
Sep 08 20:42:39.341414 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:42:39.341386    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762d711e_dc62_4d8d_9579_cd57c2195775.slice/crio-d0c71a2ee0e009df2ce0b3c13b116e9caf72bf6fffd2904ae38fcdaa1eb90038 WatchSource:0}: Error finding container d0c71a2ee0e009df2ce0b3c13b116e9caf72bf6fffd2904ae38fcdaa1eb90038: Status 404 returned error can't find the container with id d0c71a2ee0e009df2ce0b3c13b116e9caf72bf6fffd2904ae38fcdaa1eb90038
Sep 08 20:42:39.803054 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:39.803015    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k" event={"ID":"762d711e-dc62-4d8d-9579-cd57c2195775","Type":"ContainerStarted","Data":"9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c"}
Sep 08 20:42:39.803464 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:39.803086    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:39.803464 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:39.803115    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k" event={"ID":"762d711e-dc62-4d8d-9579-cd57c2195775","Type":"ContainerStarted","Data":"d0c71a2ee0e009df2ce0b3c13b116e9caf72bf6fffd2904ae38fcdaa1eb90038"}
Sep 08 20:42:39.821179 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:39.821127    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k" podStartSLOduration=1.821111042 podStartE2EDuration="1.821111042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:42:39.81875725 +0000 UTC m=+1045.983927827" watchObservedRunningTime="2026-09-08 20:42:39.821111042 +0000 UTC m=+1045.986281657"
Sep 08 20:42:50.811742 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:50.811709    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:42:56.929329 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:56.929281    2561 generic.go:356] "Generic (PLEG): container finished" podID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerID="2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14" exitCode=0
Sep 08 20:42:56.929756 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:56.929357    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" event={"ID":"dacecc0a-d25d-46c3-8e07-7e6eb89e572b","Type":"ContainerDied","Data":"2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14"}
Sep 08 20:42:56.929877 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:56.929860    2561 scope.go:117] "RemoveContainer" containerID="2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14"
Sep 08 20:42:56.930962 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:56.930928    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:56.931077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:56.931002    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:56.931077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:56.931039    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:42:57.943434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:57.943383    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" event={"ID":"dacecc0a-d25d-46c3-8e07-7e6eb89e572b","Type":"ContainerStarted","Data":"96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15"}
Sep 08 20:42:58.352155 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:58.352049    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:42:58.352402 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:58.352368    2561 patch_prober.go:28] interesting pod/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 08 20:42:58.352489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:58.352453    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 08 20:42:59.360918 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:59.360361    2561 patch_prober.go:28] interesting pod/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 08 20:42:59.360918 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:42:59.360441    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 08 20:43:00.352600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:00.352561    2561 patch_prober.go:28] interesting pod/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 08 20:43:00.352917 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:00.352867    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 08 20:43:01.352585 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:01.352130    2561 patch_prober.go:28] interesting pod/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 08 20:43:01.352585 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:01.352193    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 08 20:43:02.352682 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:02.352641    2561 patch_prober.go:28] interesting pod/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 08 20:43:02.353723 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:02.353283    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 08 20:43:03.353014 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:03.352971    2561 patch_prober.go:28] interesting pod/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 08 20:43:03.353410 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:03.353045    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 08 20:43:04.356083 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:04.356047    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:43:04.356535 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:04.356327    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:43:04.364421 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:04.364396    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:43:05.291513 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:05.291476    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"]
Sep 08 20:43:05.291766 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:05.291740    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4" podUID="8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" containerName="istio-proxy" containerID="cri-o://12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2" gracePeriod=30
Sep 08 20:43:06.290929 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.290836    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"]
Sep 08 20:43:06.295020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.295000    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"
Sep 08 20:43:06.297811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.297776    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:43:06.302619 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.302585    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"]
Sep 08 20:43:06.361258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.361227    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/892926ad-7679-4009-9e06-f0e0190b3788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6f8575-x586r\" (UID: \"892926ad-7679-4009-9e06-f0e0190b3788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"
Sep 08 20:43:06.462796 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.462757    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/892926ad-7679-4009-9e06-f0e0190b3788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6f8575-x586r\" (UID: \"892926ad-7679-4009-9e06-f0e0190b3788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"
Sep 08 20:43:06.463431 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.463408    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/892926ad-7679-4009-9e06-f0e0190b3788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6f8575-x586r\" (UID: \"892926ad-7679-4009-9e06-f0e0190b3788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"
Sep 08 20:43:06.609595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.609498    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"
Sep 08 20:43:06.885337 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:06.885309    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"]
Sep 08 20:43:06.887363 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:43:06.887323    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892926ad_7679_4009_9e06_f0e0190b3788.slice/crio-0ff14495ca91bc115b7f1a9f01d18d7de750111831f45bcb8c6649fc0eefd178 WatchSource:0}: Error finding container 0ff14495ca91bc115b7f1a9f01d18d7de750111831f45bcb8c6649fc0eefd178: Status 404 returned error can't find the container with id 0ff14495ca91bc115b7f1a9f01d18d7de750111831f45bcb8c6649fc0eefd178
Sep 08 20:43:07.014318 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:07.014280    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r" event={"ID":"892926ad-7679-4009-9e06-f0e0190b3788","Type":"ContainerStarted","Data":"0ff14495ca91bc115b7f1a9f01d18d7de750111831f45bcb8c6649fc0eefd178"}
Sep 08 20:43:08.022578 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:08.022540    2561 generic.go:356] "Generic (PLEG): container finished" podID="892926ad-7679-4009-9e06-f0e0190b3788" containerID="449f951459e8aff30af7be3b08a720adc6fc543b739a216e233ffd9764c30b75" exitCode=0
Sep 08 20:43:08.023053 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:08.022578    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r" event={"ID":"892926ad-7679-4009-9e06-f0e0190b3788","Type":"ContainerDied","Data":"449f951459e8aff30af7be3b08a720adc6fc543b739a216e233ffd9764c30b75"}
Sep 08 20:43:08.877516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:08.877477    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"]
Sep 08 20:43:08.881423 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:08.881402    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"
Sep 08 20:43:08.890074 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:08.889987    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"]
Sep 08 20:43:08.989551 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:08.989504    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca57e97-01dd-44b7-9a24-dd0a58146315-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg\" (UID: \"6ca57e97-01dd-44b7-9a24-dd0a58146315\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"
Sep 08 20:43:09.091222 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:09.091183    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca57e97-01dd-44b7-9a24-dd0a58146315-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg\" (UID: \"6ca57e97-01dd-44b7-9a24-dd0a58146315\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"
Sep 08 20:43:09.092108 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:09.091987    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca57e97-01dd-44b7-9a24-dd0a58146315-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg\" (UID: \"6ca57e97-01dd-44b7-9a24-dd0a58146315\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"
Sep 08 20:43:09.172319 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:09.172289    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"
Sep 08 20:43:09.194614 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:09.194574    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"
Sep 08 20:43:09.293756 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:09.293724    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/892926ad-7679-4009-9e06-f0e0190b3788-maas-api-service-ca\") pod \"892926ad-7679-4009-9e06-f0e0190b3788\" (UID: \"892926ad-7679-4009-9e06-f0e0190b3788\") "
Sep 08 20:43:09.294210 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:09.294182    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/892926ad-7679-4009-9e06-f0e0190b3788-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "892926ad-7679-4009-9e06-f0e0190b3788" (UID: "892926ad-7679-4009-9e06-f0e0190b3788"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:09.395401 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:09.395368    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/892926ad-7679-4009-9e06-f0e0190b3788-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:09.486580 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:09.486547    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"]
Sep 08 20:43:09.487376 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:43:09.487348    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca57e97_01dd_44b7_9a24_dd0a58146315.slice/crio-79a44f36e3cc7bf4a28dd7e02c31a4170fd5e2bf99a0b1f385a7ba81c12a9785 WatchSource:0}: Error finding container 79a44f36e3cc7bf4a28dd7e02c31a4170fd5e2bf99a0b1f385a7ba81c12a9785: Status 404 returned error can't find the container with id 79a44f36e3cc7bf4a28dd7e02c31a4170fd5e2bf99a0b1f385a7ba81c12a9785
Sep 08 20:43:10.040372 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.040259    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg" event={"ID":"6ca57e97-01dd-44b7-9a24-dd0a58146315","Type":"ContainerStarted","Data":"384fce4a5e3e8111cc1a3ec1b679758f291fcba94f39d27c5fbd6be01ec9c638"}
Sep 08 20:43:10.040534 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.040378    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg" event={"ID":"6ca57e97-01dd-44b7-9a24-dd0a58146315","Type":"ContainerStarted","Data":"79a44f36e3cc7bf4a28dd7e02c31a4170fd5e2bf99a0b1f385a7ba81c12a9785"}
Sep 08 20:43:10.041940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.041918    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r" event={"ID":"892926ad-7679-4009-9e06-f0e0190b3788","Type":"ContainerDied","Data":"0ff14495ca91bc115b7f1a9f01d18d7de750111831f45bcb8c6649fc0eefd178"}
Sep 08 20:43:10.042066 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.041954    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ff14495ca91bc115b7f1a9f01d18d7de750111831f45bcb8c6649fc0eefd178"
Sep 08 20:43:10.042066 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.041962    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"
Sep 08 20:43:10.058602 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.058551    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg" podStartSLOduration=2.058533943 podStartE2EDuration="2.058533943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:43:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:10.055140602 +0000 UTC m=+1076.220311179" watchObservedRunningTime="2026-09-08 20:43:10.058533943 +0000 UTC m=+1076.223704519"
Sep 08 20:43:10.555822 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.555751    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:43:10.608770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.608738    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-socket\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609024 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.608825    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-envoy\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609024 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.608988    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f777g\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-kube-api-access-f777g\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609024 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609020    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-data\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609170 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609048    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istiod-ca-cert\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609170 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609069    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:10.609170 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609100    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-credential-socket\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609170 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609165    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-podinfo\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609380 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609190    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-token\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609380 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609311    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:10.609380 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609353    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-certs\") pod \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\" (UID: \"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c\") "
Sep 08 20:43:10.609615 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609576    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:10.609772 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609743    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:10.609848 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609715    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:10.609930 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609874    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:10.609991 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609945    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:10.609991 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.609963    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:10.611627 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.611556    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:10.612305 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.612274    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-kube-api-access-f777g" (OuterVolumeSpecName: "kube-api-access-f777g") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "kube-api-access-f777g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:10.612459 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.612431    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:43:10.612802 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.612780    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" (UID: "8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:10.711491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.711448    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:10.711491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.711483    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f777g\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-kube-api-access-f777g\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:10.711491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.711495    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:10.711745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.711505    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:10.711745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.711514    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:10.711745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:10.711522    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:11.050580 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.050546    2561 generic.go:356] "Generic (PLEG): container finished" podID="8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" containerID="12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2" exitCode=0
Sep 08 20:43:11.050785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.050607    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"
Sep 08 20:43:11.050785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.050636    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4" event={"ID":"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c","Type":"ContainerDied","Data":"12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2"}
Sep 08 20:43:11.050785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.050697    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4" event={"ID":"8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c","Type":"ContainerDied","Data":"bdcca2e0eaa288f1c7355478c4c415e412dd4c34aba6b7001313d210950d7236"}
Sep 08 20:43:11.050785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.050722    2561 scope.go:117] "RemoveContainer" containerID="12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2"
Sep 08 20:43:11.052529 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.052503    2561 generic.go:356] "Generic (PLEG): container finished" podID="6ca57e97-01dd-44b7-9a24-dd0a58146315" containerID="384fce4a5e3e8111cc1a3ec1b679758f291fcba94f39d27c5fbd6be01ec9c638" exitCode=0
Sep 08 20:43:11.052643 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.052595    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg" event={"ID":"6ca57e97-01dd-44b7-9a24-dd0a58146315","Type":"ContainerDied","Data":"384fce4a5e3e8111cc1a3ec1b679758f291fcba94f39d27c5fbd6be01ec9c638"}
Sep 08 20:43:11.062154 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.062134    2561 scope.go:117] "RemoveContainer" containerID="12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2"
Sep 08 20:43:11.062509 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:43:11.062483    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2\": container with ID starting with 12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2 not found: ID does not exist" containerID="12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2"
Sep 08 20:43:11.062548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.062519    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2"} err="failed to get container status \"12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2\": rpc error: code = NotFound desc = could not find container \"12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2\": container with ID starting with 12dfde5694a01163688fd7d20bdbbdc067fd2850b2bdd7f1e3311037adbd7eb2 not found: ID does not exist"
Sep 08 20:43:11.084539 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.084497    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"]
Sep 08 20:43:11.090133 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:11.090102    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-458e34-openshift-default-59d55c778b-dj7j4"]
Sep 08 20:43:12.212784 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:12.212747    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"
Sep 08 20:43:12.328031 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:12.327890    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca57e97-01dd-44b7-9a24-dd0a58146315-maas-api-service-ca\") pod \"6ca57e97-01dd-44b7-9a24-dd0a58146315\" (UID: \"6ca57e97-01dd-44b7-9a24-dd0a58146315\") "
Sep 08 20:43:12.328323 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:12.328295    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ca57e97-01dd-44b7-9a24-dd0a58146315-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ca57e97-01dd-44b7-9a24-dd0a58146315" (UID: "6ca57e97-01dd-44b7-9a24-dd0a58146315"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:12.366785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:12.366743    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" path="/var/lib/kubelet/pods/8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c/volumes"
Sep 08 20:43:12.428843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:12.428804    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ca57e97-01dd-44b7-9a24-dd0a58146315-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:13.069078 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:13.069023    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"
Sep 08 20:43:13.069078 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:13.069054    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg" event={"ID":"6ca57e97-01dd-44b7-9a24-dd0a58146315","Type":"ContainerDied","Data":"79a44f36e3cc7bf4a28dd7e02c31a4170fd5e2bf99a0b1f385a7ba81c12a9785"}
Sep 08 20:43:13.069308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:13.069108    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79a44f36e3cc7bf4a28dd7e02c31a4170fd5e2bf99a0b1f385a7ba81c12a9785"
Sep 08 20:43:17.102116 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.102081    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"]
Sep 08 20:43:17.103117 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.102320    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85" podUID="063d29a9-2ba8-47f8-a8f5-46cd485e897c" containerName="payload-processing" containerID="cri-o://28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07" gracePeriod=30
Sep 08 20:43:17.109933 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.109888    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"]
Sep 08 20:43:17.110120 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.110099    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6" podUID="2dcb364f-e473-4c93-9a87-039c9902841a" containerName="payload-pre-processing" containerID="cri-o://8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec" gracePeriod=30
Sep 08 20:43:17.282055 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.282013    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"]
Sep 08 20:43:17.282539 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.282516    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6f8575-x586r"]
Sep 08 20:43:17.381547 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.381518    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:43:17.442736 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.442701    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:43:17.477884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.477845    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063d29a9-2ba8-47f8-a8f5-46cd485e897c-plugins-config-volume\") pod \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\" (UID: \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\") "
Sep 08 20:43:17.478123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.478082    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnbqm\" (UniqueName: \"kubernetes.io/projected/063d29a9-2ba8-47f8-a8f5-46cd485e897c-kube-api-access-fnbqm\") pod \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\" (UID: \"063d29a9-2ba8-47f8-a8f5-46cd485e897c\") "
Sep 08 20:43:17.478273 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.478246    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/063d29a9-2ba8-47f8-a8f5-46cd485e897c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "063d29a9-2ba8-47f8-a8f5-46cd485e897c" (UID: "063d29a9-2ba8-47f8-a8f5-46cd485e897c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:17.478431 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.478404    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063d29a9-2ba8-47f8-a8f5-46cd485e897c-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:17.480348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.480317    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063d29a9-2ba8-47f8-a8f5-46cd485e897c-kube-api-access-fnbqm" (OuterVolumeSpecName: "kube-api-access-fnbqm") pod "063d29a9-2ba8-47f8-a8f5-46cd485e897c" (UID: "063d29a9-2ba8-47f8-a8f5-46cd485e897c"). InnerVolumeSpecName "kube-api-access-fnbqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:17.579510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.579472    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dhcf\" (UniqueName: \"kubernetes.io/projected/2dcb364f-e473-4c93-9a87-039c9902841a-kube-api-access-5dhcf\") pod \"2dcb364f-e473-4c93-9a87-039c9902841a\" (UID: \"2dcb364f-e473-4c93-9a87-039c9902841a\") "
Sep 08 20:43:17.579685 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.579540    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dcb364f-e473-4c93-9a87-039c9902841a-plugins-config-volume\") pod \"2dcb364f-e473-4c93-9a87-039c9902841a\" (UID: \"2dcb364f-e473-4c93-9a87-039c9902841a\") "
Sep 08 20:43:17.579930 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.579872    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnbqm\" (UniqueName: \"kubernetes.io/projected/063d29a9-2ba8-47f8-a8f5-46cd485e897c-kube-api-access-fnbqm\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:17.579997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.579977    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dcb364f-e473-4c93-9a87-039c9902841a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2dcb364f-e473-4c93-9a87-039c9902841a" (UID: "2dcb364f-e473-4c93-9a87-039c9902841a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:17.581710 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.581683    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dcb364f-e473-4c93-9a87-039c9902841a-kube-api-access-5dhcf" (OuterVolumeSpecName: "kube-api-access-5dhcf") pod "2dcb364f-e473-4c93-9a87-039c9902841a" (UID: "2dcb364f-e473-4c93-9a87-039c9902841a"). InnerVolumeSpecName "kube-api-access-5dhcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:17.680854 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.680811    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dhcf\" (UniqueName: \"kubernetes.io/projected/2dcb364f-e473-4c93-9a87-039c9902841a-kube-api-access-5dhcf\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:17.680854 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:17.680847    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dcb364f-e473-4c93-9a87-039c9902841a-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:18.108053 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.107957    2561 generic.go:356] "Generic (PLEG): container finished" podID="063d29a9-2ba8-47f8-a8f5-46cd485e897c" containerID="28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07" exitCode=0
Sep 08 20:43:18.108609 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.108048    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"
Sep 08 20:43:18.108609 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.108085    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85" event={"ID":"063d29a9-2ba8-47f8-a8f5-46cd485e897c","Type":"ContainerDied","Data":"28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07"}
Sep 08 20:43:18.108609 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.108132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85" event={"ID":"063d29a9-2ba8-47f8-a8f5-46cd485e897c","Type":"ContainerDied","Data":"fc1f0cb71ef41341a4dd38adf68d43157828a7df5387ff6a0a2ee02f90302c08"}
Sep 08 20:43:18.108609 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.108153    2561 scope.go:117] "RemoveContainer" containerID="28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07"
Sep 08 20:43:18.110356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.110248    2561 generic.go:356] "Generic (PLEG): container finished" podID="2dcb364f-e473-4c93-9a87-039c9902841a" containerID="8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec" exitCode=0
Sep 08 20:43:18.110356 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.110320    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6" event={"ID":"2dcb364f-e473-4c93-9a87-039c9902841a","Type":"ContainerDied","Data":"8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec"}
Sep 08 20:43:18.110558 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.110361    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"
Sep 08 20:43:18.110558 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.110371    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6" event={"ID":"2dcb364f-e473-4c93-9a87-039c9902841a","Type":"ContainerDied","Data":"bfab9ca1a317fd25ce4998f7acb8834c6dec0a4088e186db1a842c3075699b2e"}
Sep 08 20:43:18.120137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.120116    2561 scope.go:117] "RemoveContainer" containerID="28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07"
Sep 08 20:43:18.120485 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:43:18.120463    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07\": container with ID starting with 28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07 not found: ID does not exist" containerID="28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07"
Sep 08 20:43:18.120599 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.120492    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07"} err="failed to get container status \"28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07\": rpc error: code = NotFound desc = could not find container \"28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07\": container with ID starting with 28c6490fc15910229a1a7ff73393a246371fe7483ebe4adba8f662967c99cc07 not found: ID does not exist"
Sep 08 20:43:18.120599 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.120508    2561 scope.go:117] "RemoveContainer" containerID="8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec"
Sep 08 20:43:18.130882 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.130862    2561 scope.go:117] "RemoveContainer" containerID="8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec"
Sep 08 20:43:18.131330 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:43:18.131306    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec\": container with ID starting with 8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec not found: ID does not exist" containerID="8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec"
Sep 08 20:43:18.131417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.131345    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec"} err="failed to get container status \"8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec\": rpc error: code = NotFound desc = could not find container \"8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec\": container with ID starting with 8c2949bf05788db92ca179267959b33179f8dfbc13920414ad20afe9ab7c6dec not found: ID does not exist"
Sep 08 20:43:18.144231 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.144190    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"]
Sep 08 20:43:18.153416 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.153385    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6f8575-6cd59f878c-bnm85"]
Sep 08 20:43:18.163118 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.163087    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"]
Sep 08 20:43:18.166928 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.166882    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f8575-55cfc498f8-h76r6"]
Sep 08 20:43:18.364927 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.364830    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="063d29a9-2ba8-47f8-a8f5-46cd485e897c" path="/var/lib/kubelet/pods/063d29a9-2ba8-47f8-a8f5-46cd485e897c/volumes"
Sep 08 20:43:18.365234 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.365219    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dcb364f-e473-4c93-9a87-039c9902841a" path="/var/lib/kubelet/pods/2dcb364f-e473-4c93-9a87-039c9902841a/volumes"
Sep 08 20:43:18.365521 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:18.365508    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="892926ad-7679-4009-9e06-f0e0190b3788" path="/var/lib/kubelet/pods/892926ad-7679-4009-9e06-f0e0190b3788/volumes"
Sep 08 20:43:19.623251 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:19.623199    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"]
Sep 08 20:43:19.623718 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:19.623481    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k" podUID="762d711e-dc62-4d8d-9579-cd57c2195775" containerName="payload-processing" containerID="cri-o://9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c" gracePeriod=30
Sep 08 20:43:19.778994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:19.778932    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"]
Sep 08 20:43:19.782838 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:19.782808    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3e6b3c9b-fp4jg"]
Sep 08 20:43:19.897640 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:19.897614    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:43:20.003544 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.003509    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltn4b\" (UniqueName: \"kubernetes.io/projected/762d711e-dc62-4d8d-9579-cd57c2195775-kube-api-access-ltn4b\") pod \"762d711e-dc62-4d8d-9579-cd57c2195775\" (UID: \"762d711e-dc62-4d8d-9579-cd57c2195775\") "
Sep 08 20:43:20.003544 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.003547    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/762d711e-dc62-4d8d-9579-cd57c2195775-plugins-config-volume\") pod \"762d711e-dc62-4d8d-9579-cd57c2195775\" (UID: \"762d711e-dc62-4d8d-9579-cd57c2195775\") "
Sep 08 20:43:20.004019 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.003993    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/762d711e-dc62-4d8d-9579-cd57c2195775-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "762d711e-dc62-4d8d-9579-cd57c2195775" (UID: "762d711e-dc62-4d8d-9579-cd57c2195775"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:20.005812 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.005788    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/762d711e-dc62-4d8d-9579-cd57c2195775-kube-api-access-ltn4b" (OuterVolumeSpecName: "kube-api-access-ltn4b") pod "762d711e-dc62-4d8d-9579-cd57c2195775" (UID: "762d711e-dc62-4d8d-9579-cd57c2195775"). InnerVolumeSpecName "kube-api-access-ltn4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:20.104824 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.104778    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltn4b\" (UniqueName: \"kubernetes.io/projected/762d711e-dc62-4d8d-9579-cd57c2195775-kube-api-access-ltn4b\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:20.104824 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.104817    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/762d711e-dc62-4d8d-9579-cd57c2195775-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:20.126674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.126634    2561 generic.go:356] "Generic (PLEG): container finished" podID="762d711e-dc62-4d8d-9579-cd57c2195775" containerID="9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c" exitCode=0
Sep 08 20:43:20.126847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.126694    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"
Sep 08 20:43:20.126847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.126725    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k" event={"ID":"762d711e-dc62-4d8d-9579-cd57c2195775","Type":"ContainerDied","Data":"9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c"}
Sep 08 20:43:20.126847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.126782    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k" event={"ID":"762d711e-dc62-4d8d-9579-cd57c2195775","Type":"ContainerDied","Data":"d0c71a2ee0e009df2ce0b3c13b116e9caf72bf6fffd2904ae38fcdaa1eb90038"}
Sep 08 20:43:20.126847 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.126808    2561 scope.go:117] "RemoveContainer" containerID="9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c"
Sep 08 20:43:20.137422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.137398    2561 scope.go:117] "RemoveContainer" containerID="9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c"
Sep 08 20:43:20.137743 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:43:20.137723    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c\": container with ID starting with 9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c not found: ID does not exist" containerID="9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c"
Sep 08 20:43:20.137796 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.137754    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c"} err="failed to get container status \"9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c\": rpc error: code = NotFound desc = could not find container \"9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c\": container with ID starting with 9327cf3ef577782602b231600997c3f2d87f36b689ca30de06f8d7e9c4567a3c not found: ID does not exist"
Sep 08 20:43:20.151723 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.151677    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"]
Sep 08 20:43:20.158405 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.158378    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3e6b3c9b-7f46d57b97-ps76k"]
Sep 08 20:43:20.364613 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.364578    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ca57e97-01dd-44b7-9a24-dd0a58146315" path="/var/lib/kubelet/pods/6ca57e97-01dd-44b7-9a24-dd0a58146315/volumes"
Sep 08 20:43:20.364992 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:20.364979    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="762d711e-dc62-4d8d-9579-cd57c2195775" path="/var/lib/kubelet/pods/762d711e-dc62-4d8d-9579-cd57c2195775/volumes"
Sep 08 20:43:23.823055 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:23.823018    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"]
Sep 08 20:43:23.823664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:23.823278    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy" containerID="cri-o://b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088" gracePeriod=30
Sep 08 20:43:24.211757 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.211723    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"]
Sep 08 20:43:24.212191 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212177    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dcb364f-e473-4c93-9a87-039c9902841a" containerName="payload-pre-processing"
Sep 08 20:43:24.212191 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212192    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dcb364f-e473-4c93-9a87-039c9902841a" containerName="payload-pre-processing"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212206    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="762d711e-dc62-4d8d-9579-cd57c2195775" containerName="payload-processing"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212213    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="762d711e-dc62-4d8d-9579-cd57c2195775" containerName="payload-processing"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212229    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="063d29a9-2ba8-47f8-a8f5-46cd485e897c" containerName="payload-processing"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212234    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="063d29a9-2ba8-47f8-a8f5-46cd485e897c" containerName="payload-processing"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212242    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" containerName="istio-proxy"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212247    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" containerName="istio-proxy"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212257    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892926ad-7679-4009-9e06-f0e0190b3788" containerName="revoke-keys"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212262    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="892926ad-7679-4009-9e06-f0e0190b3788" containerName="revoke-keys"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212269    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ca57e97-01dd-44b7-9a24-dd0a58146315" containerName="revoke-keys"
Sep 08 20:43:24.212299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212275    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ca57e97-01dd-44b7-9a24-dd0a58146315" containerName="revoke-keys"
Sep 08 20:43:24.212611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212346    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ca57e97-01dd-44b7-9a24-dd0a58146315" containerName="revoke-keys"
Sep 08 20:43:24.212611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212356    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="762d711e-dc62-4d8d-9579-cd57c2195775" containerName="payload-processing"
Sep 08 20:43:24.212611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212362    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="892926ad-7679-4009-9e06-f0e0190b3788" containerName="revoke-keys"
Sep 08 20:43:24.212611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212368    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d4bcdc3-03c7-4747-9f71-5bf95d8adb9c" containerName="istio-proxy"
Sep 08 20:43:24.212611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212376    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dcb364f-e473-4c93-9a87-039c9902841a" containerName="payload-pre-processing"
Sep 08 20:43:24.212611 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.212384    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="063d29a9-2ba8-47f8-a8f5-46cd485e897c" containerName="payload-processing"
Sep 08 20:43:24.217077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.217059    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"
Sep 08 20:43:24.219838 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.219809    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:43:24.221605 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.221580    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"]
Sep 08 20:43:24.349819 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.349784    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c0d3ee-1a97-4ef3-802c-2db82701f082-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb\" (UID: \"d2c0d3ee-1a97-4ef3-802c-2db82701f082\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"
Sep 08 20:43:24.451043 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.451001    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c0d3ee-1a97-4ef3-802c-2db82701f082-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb\" (UID: \"d2c0d3ee-1a97-4ef3-802c-2db82701f082\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"
Sep 08 20:43:24.451634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.451613    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c0d3ee-1a97-4ef3-802c-2db82701f082-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb\" (UID: \"d2c0d3ee-1a97-4ef3-802c-2db82701f082\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"
Sep 08 20:43:24.531157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.531065    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"
Sep 08 20:43:24.808280 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:24.808244    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"]
Sep 08 20:43:24.809147 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:43:24.809120    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c0d3ee_1a97_4ef3_802c_2db82701f082.slice/crio-fb620f46f0bba4dbca88e582e08f5071108bba173d6f7771266d54fb40c1e9e5 WatchSource:0}: Error finding container fb620f46f0bba4dbca88e582e08f5071108bba173d6f7771266d54fb40c1e9e5: Status 404 returned error can't find the container with id fb620f46f0bba4dbca88e582e08f5071108bba173d6f7771266d54fb40c1e9e5
Sep 08 20:43:25.162232 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:25.162198    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb" event={"ID":"d2c0d3ee-1a97-4ef3-802c-2db82701f082","Type":"ContainerStarted","Data":"85ce4408c247b25ef669003759c1031c6ef61745d06d5f9e55ea434cc5b3550b"}
Sep 08 20:43:25.162649 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:25.162249    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb" event={"ID":"d2c0d3ee-1a97-4ef3-802c-2db82701f082","Type":"ContainerStarted","Data":"fb620f46f0bba4dbca88e582e08f5071108bba173d6f7771266d54fb40c1e9e5"}
Sep 08 20:43:25.176926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:25.176853    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb" podStartSLOduration=1.17683364 podStartE2EDuration="1.17683364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:43:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:25.17534672 +0000 UTC m=+1091.340517292" watchObservedRunningTime="2026-09-08 20:43:25.17683364 +0000 UTC m=+1091.342004215"
Sep 08 20:43:26.170135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:26.170103    2561 generic.go:356] "Generic (PLEG): container finished" podID="d2c0d3ee-1a97-4ef3-802c-2db82701f082" containerID="85ce4408c247b25ef669003759c1031c6ef61745d06d5f9e55ea434cc5b3550b" exitCode=0
Sep 08 20:43:26.170500 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:26.170145    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb" event={"ID":"d2c0d3ee-1a97-4ef3-802c-2db82701f082","Type":"ContainerDied","Data":"85ce4408c247b25ef669003759c1031c6ef61745d06d5f9e55ea434cc5b3550b"}
Sep 08 20:43:26.597826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:26.597733    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"]
Sep 08 20:43:26.598085 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:26.598053    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy" containerID="cri-o://96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15" gracePeriod=30
Sep 08 20:43:27.350488 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:27.350453    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"
Sep 08 20:43:27.481680 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:27.481590    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c0d3ee-1a97-4ef3-802c-2db82701f082-maas-api-service-ca\") pod \"d2c0d3ee-1a97-4ef3-802c-2db82701f082\" (UID: \"d2c0d3ee-1a97-4ef3-802c-2db82701f082\") "
Sep 08 20:43:27.482025 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:27.482005    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2c0d3ee-1a97-4ef3-802c-2db82701f082-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2c0d3ee-1a97-4ef3-802c-2db82701f082" (UID: "d2c0d3ee-1a97-4ef3-802c-2db82701f082"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:27.583029 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:27.582994    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2c0d3ee-1a97-4ef3-802c-2db82701f082-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:28.186176 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:28.186148    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"
Sep 08 20:43:28.186363 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:28.186190    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb" event={"ID":"d2c0d3ee-1a97-4ef3-802c-2db82701f082","Type":"ContainerDied","Data":"fb620f46f0bba4dbca88e582e08f5071108bba173d6f7771266d54fb40c1e9e5"}
Sep 08 20:43:28.186363 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:28.186236    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb620f46f0bba4dbca88e582e08f5071108bba173d6f7771266d54fb40c1e9e5"
Sep 08 20:43:29.093646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.093577    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:43:29.095538 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095517    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-token\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.095670 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095567    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-podinfo\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.095670 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095588    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-data\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.095670 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095608    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dc6mt\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-kube-api-access-dc6mt\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.095830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095768    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-socket\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.095884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095834    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-credential-socket\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.095884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095862    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-certs\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.096022 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095888    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istiod-ca-cert\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.096022 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095947    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-envoy\") pod \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\" (UID: \"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc\") "
Sep 08 20:43:29.096022 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.095950    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:29.096171 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.096044    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:29.096334 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.096268    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:29.096455 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.096366    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:29.096455 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.096387    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.096455 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.096406    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.096455 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.096422    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.096701 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.096487    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:29.097884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.097859    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:43:29.097999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.097958    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:29.097999 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.097976    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-kube-api-access-dc6mt" (OuterVolumeSpecName: "kube-api-access-dc6mt") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "kube-api-access-dc6mt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:29.098095 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.098058    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" (UID: "dc2c5da4-44f0-4008-b3e8-6cfce2784dfc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:29.194989 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.194953    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerID="b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088" exitCode=0
Sep 08 20:43:29.195198 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.195022    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"
Sep 08 20:43:29.195198 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.195044    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" event={"ID":"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc","Type":"ContainerDied","Data":"b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088"}
Sep 08 20:43:29.195198 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.195104    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn" event={"ID":"dc2c5da4-44f0-4008-b3e8-6cfce2784dfc","Type":"ContainerDied","Data":"57d1e80f22cc558bcd395ea9994f40ae9ffb4e572fd1fc7d11803ab766e426e5"}
Sep 08 20:43:29.195198 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.195127    2561 scope.go:117] "RemoveContainer" containerID="b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088"
Sep 08 20:43:29.197317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.197290    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.197469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.197324    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.197469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.197339    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.197469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.197353    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.197469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.197365    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.197469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.197379    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc6mt\" (UniqueName: \"kubernetes.io/projected/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc-kube-api-access-dc6mt\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:29.206287 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.206265    2561 scope.go:117] "RemoveContainer" containerID="2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb"
Sep 08 20:43:29.215981 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.215957    2561 scope.go:117] "RemoveContainer" containerID="b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088"
Sep 08 20:43:29.216382 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:43:29.216341    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088\": container with ID starting with b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088 not found: ID does not exist" containerID="b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088"
Sep 08 20:43:29.216526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.216390    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088"} err="failed to get container status \"b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088\": rpc error: code = NotFound desc = could not find container \"b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088\": container with ID starting with b63913dc899c16e4718180e427483a59147798570165b6626b8cd0b7e9cde088 not found: ID does not exist"
Sep 08 20:43:29.216526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.216416    2561 scope.go:117] "RemoveContainer" containerID="2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb"
Sep 08 20:43:29.216744 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:43:29.216726    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb\": container with ID starting with 2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb not found: ID does not exist" containerID="2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb"
Sep 08 20:43:29.216788 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.216753    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb"} err="failed to get container status \"2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb\": rpc error: code = NotFound desc = could not find container \"2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb\": container with ID starting with 2b6223353c5bdb6b9dcffec17c22587da9a648f104d90f7569a1676d93801fcb not found: ID does not exist"
Sep 08 20:43:29.219066 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.219042    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"]
Sep 08 20:43:29.222487 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:29.222467    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6f8575-openshift-default-84484c7864-fxzkn"]
Sep 08 20:43:30.369598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:30.369562    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" path="/var/lib/kubelet/pods/dc2c5da4-44f0-4008-b3e8-6cfce2784dfc/volumes"
Sep 08 20:43:31.867752 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.867676    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:43:31.922629 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922591    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-podinfo\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.922629 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922631    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-socket\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.922842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922681    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-credential-socket\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.922842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922747    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lpt6\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-kube-api-access-8lpt6\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.922842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922780    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-token\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.922842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922820    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istiod-ca-cert\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.923076 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922884    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-data\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.923076 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922940    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-certs\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.923076 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.922970    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-envoy\") pod \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\" (UID: \"dacecc0a-d25d-46c3-8e07-7e6eb89e572b\") "
Sep 08 20:43:31.923236 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.923075    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:31.923236 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.923088    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:31.923344 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.923302    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-data" (OuterVolumeSpecName: "istio-data") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:31.923344 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.923320    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:31.923457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.923388    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:31.923457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.923409    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:31.923550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.923488    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:31.925135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.925101    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:31.925135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.925118    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-token" (OuterVolumeSpecName: "istio-token") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:31.925371 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.925352    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:43:31.925416 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:31.925354    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-kube-api-access-8lpt6" (OuterVolumeSpecName: "kube-api-access-8lpt6") pod "dacecc0a-d25d-46c3-8e07-7e6eb89e572b" (UID: "dacecc0a-d25d-46c3-8e07-7e6eb89e572b"). InnerVolumeSpecName "kube-api-access-8lpt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:32.024330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.024294    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:32.024330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.024327    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:32.024330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.024335    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:32.024330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.024344    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:32.024600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.024352    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lpt6\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-kube-api-access-8lpt6\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:32.024600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.024361    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:32.024600 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.024369    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacecc0a-d25d-46c3-8e07-7e6eb89e572b-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:43:32.218513 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.218475    2561 generic.go:356] "Generic (PLEG): container finished" podID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerID="96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15" exitCode=0
Sep 08 20:43:32.218690 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.218527    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"
Sep 08 20:43:32.218690 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.218526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" event={"ID":"dacecc0a-d25d-46c3-8e07-7e6eb89e572b","Type":"ContainerDied","Data":"96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15"}
Sep 08 20:43:32.218690 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.218649    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x" event={"ID":"dacecc0a-d25d-46c3-8e07-7e6eb89e572b","Type":"ContainerDied","Data":"0658f21172ebec2dc2fbe36fdbb7982570681a8466fff9cf41d4e48483fe77d6"}
Sep 08 20:43:32.218690 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.218679    2561 scope.go:117] "RemoveContainer" containerID="96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15"
Sep 08 20:43:32.229033 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.229014    2561 scope.go:117] "RemoveContainer" containerID="2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14"
Sep 08 20:43:32.238796 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.238771    2561 scope.go:117] "RemoveContainer" containerID="96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15"
Sep 08 20:43:32.239429 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:43:32.239397    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15\": container with ID starting with 96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15 not found: ID does not exist" containerID="96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15"
Sep 08 20:43:32.239576 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.239440    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15"} err="failed to get container status \"96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15\": rpc error: code = NotFound desc = could not find container \"96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15\": container with ID starting with 96063c3f9d174535a49a22269d8993ed13efb9731f0abd2f6379498e24a7bf15 not found: ID does not exist"
Sep 08 20:43:32.239576 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.239467    2561 scope.go:117] "RemoveContainer" containerID="2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14"
Sep 08 20:43:32.239986 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:43:32.239962    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14\": container with ID starting with 2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14 not found: ID does not exist" containerID="2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14"
Sep 08 20:43:32.240065 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.239994    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14"} err="failed to get container status \"2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14\": rpc error: code = NotFound desc = could not find container \"2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14\": container with ID starting with 2eb110a098432212b600971d0ae9e86a657895378fe3d60d5979329ce03c9d14 not found: ID does not exist"
Sep 08 20:43:32.242736 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.242649    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"]
Sep 08 20:43:32.247235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.247202    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3e6b3c9b-openshift-default-7c59fc6d96-jcz4x"]
Sep 08 20:43:32.364735 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:32.364696    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" path="/var/lib/kubelet/pods/dacecc0a-d25d-46c3-8e07-7e6eb89e572b/volumes"
Sep 08 20:43:34.909297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:34.909260    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"]
Sep 08 20:43:34.917506 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:34.914568    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a6137b-k86mb"]
Sep 08 20:43:36.363911 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:36.363857    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2c0d3ee-1a97-4ef3-802c-2db82701f082" path="/var/lib/kubelet/pods/d2c0d3ee-1a97-4ef3-802c-2db82701f082/volumes"
Sep 08 20:43:40.121245 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121094    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"]
Sep 08 20:43:40.121781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121758    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy"
Sep 08 20:43:40.121867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121783    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy"
Sep 08 20:43:40.121867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121832    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy"
Sep 08 20:43:40.121867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121841    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy"
Sep 08 20:43:40.121867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121852    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c0d3ee-1a97-4ef3-802c-2db82701f082" containerName="revoke-keys"
Sep 08 20:43:40.121867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121861    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c0d3ee-1a97-4ef3-802c-2db82701f082" containerName="revoke-keys"
Sep 08 20:43:40.122148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121873    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy"
Sep 08 20:43:40.122148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121881    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy"
Sep 08 20:43:40.122148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121920    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy"
Sep 08 20:43:40.122148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.121930    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy"
Sep 08 20:43:40.122148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.122057    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy"
Sep 08 20:43:40.122148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.122072    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy"
Sep 08 20:43:40.122148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.122082    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2c0d3ee-1a97-4ef3-802c-2db82701f082" containerName="revoke-keys"
Sep 08 20:43:40.127185 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.127159    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.129842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.129821    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-27d4c1-openshift-default-dockercfg-59kl2\""
Sep 08 20:43:40.140008 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.139981    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"]
Sep 08 20:43:40.202009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.201966    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd571727-7e02-4cca-8725-6d7a3489bf68-istiod-ca-cert\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.202009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.202013    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-credential-socket\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.202294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.202092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-socket\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.202294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.202132    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-podinfo\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.202294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.202200    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-token\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.202294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.202237    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-certs\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.202294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.202265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-envoy\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.202516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.202302    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-data\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.202516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.202355    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stw9j\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-kube-api-access-stw9j\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.303820 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.303778    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-token\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.303820 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.303820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-certs\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304112 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.303840    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-envoy\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304112 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.303873    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-data\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304112 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.303938    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stw9j\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-kube-api-access-stw9j\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304112 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304048    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd571727-7e02-4cca-8725-6d7a3489bf68-istiod-ca-cert\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304112 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304073    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-credential-socket\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304132    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-socket\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304166    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-podinfo\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304322    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-certs\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304358    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-data\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304584 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304506    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-credential-socket\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304649 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304584    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-socket\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.304701 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.304673    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd571727-7e02-4cca-8725-6d7a3489bf68-istiod-ca-cert\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.306303 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.306285    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-envoy\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.306595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.306575    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-podinfo\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.311494 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.311471    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-token\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.311645 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.311626    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stw9j\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-kube-api-access-stw9j\") pod \"e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") " pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.440838 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.440794    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:40.729512 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.729484    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"]
Sep 08 20:43:40.731681 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:43:40.731650    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd571727_7e02_4cca_8725_6d7a3489bf68.slice/crio-779393c6b9679008ac8aedf0385a65d0a97b2384449295800434d3ac1951c192 WatchSource:0}: Error finding container 779393c6b9679008ac8aedf0385a65d0a97b2384449295800434d3ac1951c192: Status 404 returned error can't find the container with id 779393c6b9679008ac8aedf0385a65d0a97b2384449295800434d3ac1951c192
Sep 08 20:43:40.734399 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.734357    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:43:40.734485 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.734449    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:43:40.734533 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:40.734492    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:43:41.280359 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:41.280311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" event={"ID":"cd571727-7e02-4cca-8725-6d7a3489bf68","Type":"ContainerStarted","Data":"7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec"}
Sep 08 20:43:41.280772 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:41.280418    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" event={"ID":"cd571727-7e02-4cca-8725-6d7a3489bf68","Type":"ContainerStarted","Data":"779393c6b9679008ac8aedf0385a65d0a97b2384449295800434d3ac1951c192"}
Sep 08 20:43:41.303079 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:41.303020    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" podStartSLOduration=1.30300302 podStartE2EDuration="1.30300302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:43:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:41.299824335 +0000 UTC m=+1107.464994912" watchObservedRunningTime="2026-09-08 20:43:41.30300302 +0000 UTC m=+1107.468173597"
Sep 08 20:43:41.441009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:41.440960    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:41.446744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:41.446716    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:42.286584 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:42.286551    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:42.287744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:42.287726    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:43:45.216396 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.216363    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"]
Sep 08 20:43:45.216903 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.216879    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dacecc0a-d25d-46c3-8e07-7e6eb89e572b" containerName="istio-proxy"
Sep 08 20:43:45.216948 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.216919    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc2c5da4-44f0-4008-b3e8-6cfce2784dfc" containerName="istio-proxy"
Sep 08 20:43:45.220018 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.220000    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:45.222256 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.222235    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-27d4c1\""
Sep 08 20:43:45.222363 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.222237    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-27d4c1-dockercfg-8dzmv\""
Sep 08 20:43:45.231589 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.231517    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"]
Sep 08 20:43:45.242714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.242684    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"]
Sep 08 20:43:45.246349 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.246328    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:45.260344 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.260314    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"]
Sep 08 20:43:45.352820 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.352789    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skkvv\" (UniqueName: \"kubernetes.io/projected/f38d5233-b15b-4507-ace3-45f6479f4584-kube-api-access-skkvv\") pod \"payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs\" (UID: \"f38d5233-b15b-4507-ace3-45f6479f4584\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:45.353033 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.352876    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f38d5233-b15b-4507-ace3-45f6479f4584-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs\" (UID: \"f38d5233-b15b-4507-ace3-45f6479f4584\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:45.353033 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.352952    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4djhg\" (UniqueName: \"kubernetes.io/projected/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-kube-api-access-4djhg\") pod \"payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm\" (UID: \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:45.353033 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.353007    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm\" (UID: \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:45.454374 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.454336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f38d5233-b15b-4507-ace3-45f6479f4584-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs\" (UID: \"f38d5233-b15b-4507-ace3-45f6479f4584\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:45.454546 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.454385    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4djhg\" (UniqueName: \"kubernetes.io/projected/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-kube-api-access-4djhg\") pod \"payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm\" (UID: \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:45.454546 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.454421    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm\" (UID: \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:45.454546 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.454469    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skkvv\" (UniqueName: \"kubernetes.io/projected/f38d5233-b15b-4507-ace3-45f6479f4584-kube-api-access-skkvv\") pod \"payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs\" (UID: \"f38d5233-b15b-4507-ace3-45f6479f4584\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:45.455353 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.455329    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm\" (UID: \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:45.455401 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.455329    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f38d5233-b15b-4507-ace3-45f6479f4584-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs\" (UID: \"f38d5233-b15b-4507-ace3-45f6479f4584\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:45.462873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.462850    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skkvv\" (UniqueName: \"kubernetes.io/projected/f38d5233-b15b-4507-ace3-45f6479f4584-kube-api-access-skkvv\") pod \"payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs\" (UID: \"f38d5233-b15b-4507-ace3-45f6479f4584\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:45.463072 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.463051    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4djhg\" (UniqueName: \"kubernetes.io/projected/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-kube-api-access-4djhg\") pod \"payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm\" (UID: \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:45.533350 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.533260    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:45.559737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.559703    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:45.813837 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.813803    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"]
Sep 08 20:43:45.814083 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:43:45.814056    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38d5233_b15b_4507_ace3_45f6479f4584.slice/crio-c72e06367d899baa6663c7d5500245931565a7d2bbfb254ce2e4947a6b7c6417 WatchSource:0}: Error finding container c72e06367d899baa6663c7d5500245931565a7d2bbfb254ce2e4947a6b7c6417: Status 404 returned error can't find the container with id c72e06367d899baa6663c7d5500245931565a7d2bbfb254ce2e4947a6b7c6417
Sep 08 20:43:45.846690 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:45.846662    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"]
Sep 08 20:43:45.849237 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:43:45.849206    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a458b7_0de8_41d3_80a7_6cf3f491f1ce.slice/crio-196e741b770c8bc616fd833b9b0c528891158b11d725d16a8c70828b59d4fc80 WatchSource:0}: Error finding container 196e741b770c8bc616fd833b9b0c528891158b11d725d16a8c70828b59d4fc80: Status 404 returned error can't find the container with id 196e741b770c8bc616fd833b9b0c528891158b11d725d16a8c70828b59d4fc80
Sep 08 20:43:46.314101 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:46.314049    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs" event={"ID":"f38d5233-b15b-4507-ace3-45f6479f4584","Type":"ContainerStarted","Data":"e960ff300a4ce98b821cf479c49d77fa45a469277644e118a74a40bbc97f7a19"}
Sep 08 20:43:46.314595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:46.314256    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:46.314595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:46.314304    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs" event={"ID":"f38d5233-b15b-4507-ace3-45f6479f4584","Type":"ContainerStarted","Data":"c72e06367d899baa6663c7d5500245931565a7d2bbfb254ce2e4947a6b7c6417"}
Sep 08 20:43:46.315724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:46.315704    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm" event={"ID":"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce","Type":"ContainerStarted","Data":"a7575beb716fb2c37f462173756a1be32583a242104b0e61a3d47d33d5f596ac"}
Sep 08 20:43:46.315807 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:46.315738    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm" event={"ID":"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce","Type":"ContainerStarted","Data":"196e741b770c8bc616fd833b9b0c528891158b11d725d16a8c70828b59d4fc80"}
Sep 08 20:43:46.315807 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:46.315758    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:43:46.334259 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:46.334169    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs" podStartSLOduration=1.334146916 podStartE2EDuration="1.334146916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:43:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:46.330813767 +0000 UTC m=+1112.495984342" watchObservedRunningTime="2026-09-08 20:43:46.334146916 +0000 UTC m=+1112.499317494"
Sep 08 20:43:46.389649 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:46.389596    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm" podStartSLOduration=1.38958189 podStartE2EDuration="1.38958189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:43:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:46.386458269 +0000 UTC m=+1112.551628847" watchObservedRunningTime="2026-09-08 20:43:46.38958189 +0000 UTC m=+1112.554752466"
Sep 08 20:43:47.864797 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:47.864759    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"]
Sep 08 20:43:47.868738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:47.868720    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:47.871343 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:47.871320    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-78dd9f\""
Sep 08 20:43:47.881231 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:47.881199    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"]
Sep 08 20:43:47.980980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:47.980943    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2\" (UID: \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:47.981205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:47.981168    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bmk6\" (UniqueName: \"kubernetes.io/projected/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-kube-api-access-7bmk6\") pod \"payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2\" (UID: \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:48.081913 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:48.081862    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bmk6\" (UniqueName: \"kubernetes.io/projected/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-kube-api-access-7bmk6\") pod \"payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2\" (UID: \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:48.082114 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:48.081934    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2\" (UID: \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:48.082657 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:48.082625    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2\" (UID: \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:48.090276 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:48.090248    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bmk6\" (UniqueName: \"kubernetes.io/projected/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-kube-api-access-7bmk6\") pod \"payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2\" (UID: \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:48.183018 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:48.182974    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:48.464878 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:43:48.464848    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66ca5239_bcc9_4409_9eb5_308d0fa61cb3.slice/crio-23581f26e153b01e98682478e2bfa21fc4bde0b81452ab491993b93bc8b0dab2 WatchSource:0}: Error finding container 23581f26e153b01e98682478e2bfa21fc4bde0b81452ab491993b93bc8b0dab2: Status 404 returned error can't find the container with id 23581f26e153b01e98682478e2bfa21fc4bde0b81452ab491993b93bc8b0dab2
Sep 08 20:43:48.464878 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:48.464884    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"]
Sep 08 20:43:49.342001 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:49.341961    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2" event={"ID":"66ca5239-bcc9-4409-9eb5-308d0fa61cb3","Type":"ContainerStarted","Data":"3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27"}
Sep 08 20:43:49.342418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:49.342024    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2" event={"ID":"66ca5239-bcc9-4409-9eb5-308d0fa61cb3","Type":"ContainerStarted","Data":"23581f26e153b01e98682478e2bfa21fc4bde0b81452ab491993b93bc8b0dab2"}
Sep 08 20:43:49.342418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:49.342053    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:43:49.360214 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:49.360153    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2" podStartSLOduration=2.360137055 podStartE2EDuration="2.360137055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:43:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:49.357768865 +0000 UTC m=+1115.522939441" watchObservedRunningTime="2026-09-08 20:43:49.360137055 +0000 UTC m=+1115.525307630"
Sep 08 20:43:57.325369 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:57.325337    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:43:57.325859 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:43:57.325704    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:44:00.351882 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:00.351847    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:44:02.433417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:02.433380    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerID="7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec" exitCode=0
Sep 08 20:44:02.433856 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:02.433455    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" event={"ID":"cd571727-7e02-4cca-8725-6d7a3489bf68","Type":"ContainerDied","Data":"7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec"}
Sep 08 20:44:02.433926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:02.433910    2561 scope.go:117] "RemoveContainer" containerID="7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec"
Sep 08 20:44:02.435147 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:02.435116    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:44:02.435222 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:02.435193    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:44:02.435261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:02.435226    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:44:03.460010 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:03.459967    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" event={"ID":"cd571727-7e02-4cca-8725-6d7a3489bf68","Type":"ContainerStarted","Data":"a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087"}
Sep 08 20:44:04.441576 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:04.441537    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:44:04.442200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:04.441833    2561 patch_prober.go:28] interesting pod/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 08 20:44:04.442200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:04.441909    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 08 20:44:05.441345 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:05.441298    2561 patch_prober.go:28] interesting pod/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 08 20:44:05.441977 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:05.441942    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 08 20:44:06.441875 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:06.441832    2561 patch_prober.go:28] interesting pod/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 08 20:44:06.442523 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:06.442481    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 08 20:44:07.441450 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:07.441393    2561 patch_prober.go:28] interesting pod/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 08 20:44:07.441674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:07.441498    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 08 20:44:08.442282 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:08.442245    2561 patch_prober.go:28] interesting pod/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 08 20:44:08.443327 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:08.442865    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 08 20:44:09.446296 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:09.446260    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:44:09.446714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:09.446682    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:44:09.447362 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:09.447340    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:44:14.094644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.094608    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"]
Sep 08 20:44:14.098344 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.098327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:14.100714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.100691    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-74f9a1\""
Sep 08 20:44:14.109644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.109623    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"]
Sep 08 20:44:14.143495 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.143459    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"]
Sep 08 20:44:14.146994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.146976    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:14.149769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.149742    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-74f9a1-dockercfg-cv42t\""
Sep 08 20:44:14.160491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.160468    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"]
Sep 08 20:44:14.165090 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.165066    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cca8bf38-44d5-491a-83be-adfbbd61600d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg\" (UID: \"cca8bf38-44d5-491a-83be-adfbbd61600d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:14.165214 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.165173    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2gh8\" (UniqueName: \"kubernetes.io/projected/cca8bf38-44d5-491a-83be-adfbbd61600d-kube-api-access-v2gh8\") pod \"payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg\" (UID: \"cca8bf38-44d5-491a-83be-adfbbd61600d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:14.266462 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.266428    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2gh8\" (UniqueName: \"kubernetes.io/projected/cca8bf38-44d5-491a-83be-adfbbd61600d-kube-api-access-v2gh8\") pod \"payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg\" (UID: \"cca8bf38-44d5-491a-83be-adfbbd61600d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:14.266665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.266493    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkxn2\" (UniqueName: \"kubernetes.io/projected/9564f612-bac0-4c19-ba35-a882800c253c-kube-api-access-kkxn2\") pod \"payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr\" (UID: \"9564f612-bac0-4c19-ba35-a882800c253c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:14.266665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.266553    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cca8bf38-44d5-491a-83be-adfbbd61600d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg\" (UID: \"cca8bf38-44d5-491a-83be-adfbbd61600d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:14.266665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.266597    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9564f612-bac0-4c19-ba35-a882800c253c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr\" (UID: \"9564f612-bac0-4c19-ba35-a882800c253c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:14.269163 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.269136    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-74f9a1\""
Sep 08 20:44:14.277790 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.277761    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cca8bf38-44d5-491a-83be-adfbbd61600d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg\" (UID: \"cca8bf38-44d5-491a-83be-adfbbd61600d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:14.283591 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.283557    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2gh8\" (UniqueName: \"kubernetes.io/projected/cca8bf38-44d5-491a-83be-adfbbd61600d-kube-api-access-v2gh8\") pod \"payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg\" (UID: \"cca8bf38-44d5-491a-83be-adfbbd61600d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:14.368020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.367926    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9564f612-bac0-4c19-ba35-a882800c253c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr\" (UID: \"9564f612-bac0-4c19-ba35-a882800c253c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:14.368348 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.368327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxn2\" (UniqueName: \"kubernetes.io/projected/9564f612-bac0-4c19-ba35-a882800c253c-kube-api-access-kkxn2\") pod \"payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr\" (UID: \"9564f612-bac0-4c19-ba35-a882800c253c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:14.369056 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.369033    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9564f612-bac0-4c19-ba35-a882800c253c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr\" (UID: \"9564f612-bac0-4c19-ba35-a882800c253c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:14.376735 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.376704    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkxn2\" (UniqueName: \"kubernetes.io/projected/9564f612-bac0-4c19-ba35-a882800c253c-kube-api-access-kkxn2\") pod \"payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr\" (UID: \"9564f612-bac0-4c19-ba35-a882800c253c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:14.410732 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.410699    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:14.460778 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.460744    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-74f9a1-dockercfg-cv42t\""
Sep 08 20:44:14.469038 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.468932    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:14.700727 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.700705    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"]
Sep 08 20:44:14.701202 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:14.701180    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca8bf38_44d5_491a_83be_adfbbd61600d.slice/crio-21c582f5f9e0b97c36c526f0aed9048f0ae505751ee95b5fe049bb8b68862215 WatchSource:0}: Error finding container 21c582f5f9e0b97c36c526f0aed9048f0ae505751ee95b5fe049bb8b68862215: Status 404 returned error can't find the container with id 21c582f5f9e0b97c36c526f0aed9048f0ae505751ee95b5fe049bb8b68862215
Sep 08 20:44:14.788142 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:14.788116    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"]
Sep 08 20:44:14.789288 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:14.789259    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9564f612_bac0_4c19_ba35_a882800c253c.slice/crio-9440fb7da9562ad502512b08feb464078a2dcb0d13382256a2ff63fe27f141d3 WatchSource:0}: Error finding container 9440fb7da9562ad502512b08feb464078a2dcb0d13382256a2ff63fe27f141d3: Status 404 returned error can't find the container with id 9440fb7da9562ad502512b08feb464078a2dcb0d13382256a2ff63fe27f141d3
Sep 08 20:44:15.558714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:15.558672    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg" event={"ID":"cca8bf38-44d5-491a-83be-adfbbd61600d","Type":"ContainerStarted","Data":"1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119"}
Sep 08 20:44:15.559207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:15.558742    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg" event={"ID":"cca8bf38-44d5-491a-83be-adfbbd61600d","Type":"ContainerStarted","Data":"21c582f5f9e0b97c36c526f0aed9048f0ae505751ee95b5fe049bb8b68862215"}
Sep 08 20:44:15.559207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:15.558770    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:15.560773 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:15.560747    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr" event={"ID":"9564f612-bac0-4c19-ba35-a882800c253c","Type":"ContainerStarted","Data":"9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7"}
Sep 08 20:44:15.560943 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:15.560790    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:15.560943 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:15.560807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr" event={"ID":"9564f612-bac0-4c19-ba35-a882800c253c","Type":"ContainerStarted","Data":"9440fb7da9562ad502512b08feb464078a2dcb0d13382256a2ff63fe27f141d3"}
Sep 08 20:44:15.581468 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:15.581407    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg" podStartSLOduration=1.581389744 podStartE2EDuration="1.581389744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:15.579073001 +0000 UTC m=+1141.744243578" watchObservedRunningTime="2026-09-08 20:44:15.581389744 +0000 UTC m=+1141.746560319"
Sep 08 20:44:15.603622 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:15.603571    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr" podStartSLOduration=1.603552174 podStartE2EDuration="1.603552174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:15.602987645 +0000 UTC m=+1141.768158223" watchObservedRunningTime="2026-09-08 20:44:15.603552174 +0000 UTC m=+1141.768722755"
Sep 08 20:44:16.217007 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.216975    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"]
Sep 08 20:44:16.221053 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.221024    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:16.227686 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.227656    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-c9590b-dockercfg-xm6l2\""
Sep 08 20:44:16.227835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.227665    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-c9590b\""
Sep 08 20:44:16.237089 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.237062    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"]
Sep 08 20:44:16.286376 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.286346    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55102c6e-2363-428b-8f42-2d47f1638e72-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h\" (UID: \"55102c6e-2363-428b-8f42-2d47f1638e72\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:16.286551 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.286391    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhrt8\" (UniqueName: \"kubernetes.io/projected/55102c6e-2363-428b-8f42-2d47f1638e72-kube-api-access-xhrt8\") pod \"payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h\" (UID: \"55102c6e-2363-428b-8f42-2d47f1638e72\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:16.387606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.387571    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55102c6e-2363-428b-8f42-2d47f1638e72-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h\" (UID: \"55102c6e-2363-428b-8f42-2d47f1638e72\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:16.387606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.387620    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhrt8\" (UniqueName: \"kubernetes.io/projected/55102c6e-2363-428b-8f42-2d47f1638e72-kube-api-access-xhrt8\") pod \"payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h\" (UID: \"55102c6e-2363-428b-8f42-2d47f1638e72\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:16.388381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.388361    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55102c6e-2363-428b-8f42-2d47f1638e72-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h\" (UID: \"55102c6e-2363-428b-8f42-2d47f1638e72\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:16.395818 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.395791    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhrt8\" (UniqueName: \"kubernetes.io/projected/55102c6e-2363-428b-8f42-2d47f1638e72-kube-api-access-xhrt8\") pod \"payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h\" (UID: \"55102c6e-2363-428b-8f42-2d47f1638e72\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:16.533339 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.533249    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:16.821275 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:16.821250    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"]
Sep 08 20:44:16.822880 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:16.822850    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55102c6e_2363_428b_8f42_2d47f1638e72.slice/crio-a7ed85307ec4241e947455f2c4726abb28d3af3f04aa17e24f8c6819f12f9637 WatchSource:0}: Error finding container a7ed85307ec4241e947455f2c4726abb28d3af3f04aa17e24f8c6819f12f9637: Status 404 returned error can't find the container with id a7ed85307ec4241e947455f2c4726abb28d3af3f04aa17e24f8c6819f12f9637
Sep 08 20:44:17.581151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:17.581100    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h" event={"ID":"55102c6e-2363-428b-8f42-2d47f1638e72","Type":"ContainerStarted","Data":"a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e"}
Sep 08 20:44:17.581368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:17.581164    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h" event={"ID":"55102c6e-2363-428b-8f42-2d47f1638e72","Type":"ContainerStarted","Data":"a7ed85307ec4241e947455f2c4726abb28d3af3f04aa17e24f8c6819f12f9637"}
Sep 08 20:44:17.581368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:17.581193    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:17.600107 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:17.600056    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h" podStartSLOduration=1.600040575 podStartE2EDuration="1.600040575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:17.597745793 +0000 UTC m=+1143.762916371" watchObservedRunningTime="2026-09-08 20:44:17.600040575 +0000 UTC m=+1143.765211184"
Sep 08 20:44:26.574004 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:26.573971    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:44:26.574379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:26.574132    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:44:28.590961 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:28.590890    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:44:37.879839 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:37.879743    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:37.885207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:37.885171    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:37.888849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:37.887928    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qqm77\""
Sep 08 20:44:37.890121 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:37.890092    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:38.005580 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:38.005534    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbrsj\" (UniqueName: \"kubernetes.io/projected/6fc14419-2b1e-4e15-bb35-247a57b7b8db-kube-api-access-pbrsj\") pod \"test-curl-7701-139756345206512\" (UID: \"6fc14419-2b1e-4e15-bb35-247a57b7b8db\") " pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:38.107037 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:38.106999    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbrsj\" (UniqueName: \"kubernetes.io/projected/6fc14419-2b1e-4e15-bb35-247a57b7b8db-kube-api-access-pbrsj\") pod \"test-curl-7701-139756345206512\" (UID: \"6fc14419-2b1e-4e15-bb35-247a57b7b8db\") " pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:38.115710 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:38.115680    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbrsj\" (UniqueName: \"kubernetes.io/projected/6fc14419-2b1e-4e15-bb35-247a57b7b8db-kube-api-access-pbrsj\") pod \"test-curl-7701-139756345206512\" (UID: \"6fc14419-2b1e-4e15-bb35-247a57b7b8db\") " pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:38.197798 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:38.197758    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:38.476359 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:38.476332    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:38.477793 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:38.477751    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fc14419_2b1e_4e15_bb35_247a57b7b8db.slice/crio-cdb760792d7f34f86e23e00180c56e304b0aabf7859b5463b8b752748b9c718c WatchSource:0}: Error finding container cdb760792d7f34f86e23e00180c56e304b0aabf7859b5463b8b752748b9c718c: Status 404 returned error can't find the container with id cdb760792d7f34f86e23e00180c56e304b0aabf7859b5463b8b752748b9c718c
Sep 08 20:44:38.479271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:38.479250    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:44:38.726419 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:38.726321    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-139756345206512" event={"ID":"6fc14419-2b1e-4e15-bb35-247a57b7b8db","Type":"ContainerStarted","Data":"cdb760792d7f34f86e23e00180c56e304b0aabf7859b5463b8b752748b9c718c"}
Sep 08 20:44:39.733936 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:39.733886    2561 generic.go:356] "Generic (PLEG): container finished" podID="6fc14419-2b1e-4e15-bb35-247a57b7b8db" containerID="3f2a2aed96ac18a9c0fa296af4e9b59b2853f4172e3fc051b0c3b42ff8a7a1dc" exitCode=0
Sep 08 20:44:39.734351 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:39.733928    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-139756345206512" event={"ID":"6fc14419-2b1e-4e15-bb35-247a57b7b8db","Type":"ContainerDied","Data":"3f2a2aed96ac18a9c0fa296af4e9b59b2853f4172e3fc051b0c3b42ff8a7a1dc"}
Sep 08 20:44:40.889227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:40.889202    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:40.922219 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:40.922188    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7701-139756345206512_6fc14419-2b1e-4e15-bb35-247a57b7b8db/test-curl-7701-139756345206512/0.log"
Sep 08 20:44:40.947381 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:40.947344    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:40.950927 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:40.950881    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:41.036452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.036353    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbrsj\" (UniqueName: \"kubernetes.io/projected/6fc14419-2b1e-4e15-bb35-247a57b7b8db-kube-api-access-pbrsj\") pod \"6fc14419-2b1e-4e15-bb35-247a57b7b8db\" (UID: \"6fc14419-2b1e-4e15-bb35-247a57b7b8db\") "
Sep 08 20:44:41.038538 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.038508    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fc14419-2b1e-4e15-bb35-247a57b7b8db-kube-api-access-pbrsj" (OuterVolumeSpecName: "kube-api-access-pbrsj") pod "6fc14419-2b1e-4e15-bb35-247a57b7b8db" (UID: "6fc14419-2b1e-4e15-bb35-247a57b7b8db"). InnerVolumeSpecName "kube-api-access-pbrsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:44:41.075510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.075469    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:41.075978 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.075963    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fc14419-2b1e-4e15-bb35-247a57b7b8db" containerName="test-curl-7701-139756345206512"
Sep 08 20:44:41.076034 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.075980    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fc14419-2b1e-4e15-bb35-247a57b7b8db" containerName="test-curl-7701-139756345206512"
Sep 08 20:44:41.076071 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.076065    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fc14419-2b1e-4e15-bb35-247a57b7b8db" containerName="test-curl-7701-139756345206512"
Sep 08 20:44:41.079244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.079226    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:41.086233 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.086203    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:41.137661 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.137620    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbrsj\" (UniqueName: \"kubernetes.io/projected/6fc14419-2b1e-4e15-bb35-247a57b7b8db-kube-api-access-pbrsj\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:44:41.239097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.239054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6s9p\" (UniqueName: \"kubernetes.io/projected/e6277531-b656-4e8e-befa-70935d02010d-kube-api-access-x6s9p\") pod \"test-curl-7701-139756345206512\" (UID: \"e6277531-b656-4e8e-befa-70935d02010d\") " pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:41.340113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.340020    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6s9p\" (UniqueName: \"kubernetes.io/projected/e6277531-b656-4e8e-befa-70935d02010d-kube-api-access-x6s9p\") pod \"test-curl-7701-139756345206512\" (UID: \"e6277531-b656-4e8e-befa-70935d02010d\") " pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:41.349010 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.348961    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6s9p\" (UniqueName: \"kubernetes.io/projected/e6277531-b656-4e8e-befa-70935d02010d-kube-api-access-x6s9p\") pod \"test-curl-7701-139756345206512\" (UID: \"e6277531-b656-4e8e-befa-70935d02010d\") " pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:41.389957 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.389878    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:41.664107 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.664074    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:41.665431 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:41.665397    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6277531_b656_4e8e_befa_70935d02010d.slice/crio-33d3c4df41febb655643502cbd4ebc19580b2b0cf8ad8a99ca9f64ecfd3f44e2 WatchSource:0}: Error finding container 33d3c4df41febb655643502cbd4ebc19580b2b0cf8ad8a99ca9f64ecfd3f44e2: Status 404 returned error can't find the container with id 33d3c4df41febb655643502cbd4ebc19580b2b0cf8ad8a99ca9f64ecfd3f44e2
Sep 08 20:44:41.748860 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.748824    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-139756345206512" event={"ID":"e6277531-b656-4e8e-befa-70935d02010d","Type":"ContainerStarted","Data":"33d3c4df41febb655643502cbd4ebc19580b2b0cf8ad8a99ca9f64ecfd3f44e2"}
Sep 08 20:44:41.750255 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.750234    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:41.750255 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.750250    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdb760792d7f34f86e23e00180c56e304b0aabf7859b5463b8b752748b9c718c"
Sep 08 20:44:41.761998 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:41.761945    2561 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7701-139756345206512" oldPodUID="6fc14419-2b1e-4e15-bb35-247a57b7b8db" podUID="e6277531-b656-4e8e-befa-70935d02010d"
Sep 08 20:44:42.368098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:42.368066    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fc14419-2b1e-4e15-bb35-247a57b7b8db" path="/var/lib/kubelet/pods/6fc14419-2b1e-4e15-bb35-247a57b7b8db/volumes"
Sep 08 20:44:42.758747 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:42.758708    2561 generic.go:356] "Generic (PLEG): container finished" podID="e6277531-b656-4e8e-befa-70935d02010d" containerID="31ac6290157dc0d267912d28759630fab8c6256a7b46e33c2075e9fa1e7cd452" exitCode=0
Sep 08 20:44:42.758986 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:42.758786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-139756345206512" event={"ID":"e6277531-b656-4e8e-befa-70935d02010d","Type":"ContainerDied","Data":"31ac6290157dc0d267912d28759630fab8c6256a7b46e33c2075e9fa1e7cd452"}
Sep 08 20:44:43.907779 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:43.907750    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:43.927983 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:43.927955    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7701-139756345206512_e6277531-b656-4e8e-befa-70935d02010d/test-curl-7701-139756345206512/0.log"
Sep 08 20:44:43.951342 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:43.951304    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:43.954178 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:43.954152    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7701-139756345206512"]
Sep 08 20:44:43.960506 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:43.960476    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6s9p\" (UniqueName: \"kubernetes.io/projected/e6277531-b656-4e8e-befa-70935d02010d-kube-api-access-x6s9p\") pod \"e6277531-b656-4e8e-befa-70935d02010d\" (UID: \"e6277531-b656-4e8e-befa-70935d02010d\") "
Sep 08 20:44:43.962613 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:43.962590    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6277531-b656-4e8e-befa-70935d02010d-kube-api-access-x6s9p" (OuterVolumeSpecName: "kube-api-access-x6s9p") pod "e6277531-b656-4e8e-befa-70935d02010d" (UID: "e6277531-b656-4e8e-befa-70935d02010d"). InnerVolumeSpecName "kube-api-access-x6s9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:44:44.062152 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.062062    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6s9p\" (UniqueName: \"kubernetes.io/projected/e6277531-b656-4e8e-befa-70935d02010d-kube-api-access-x6s9p\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:44:44.073862 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.073828    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:44.074360 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.074345    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6277531-b656-4e8e-befa-70935d02010d" containerName="test-curl-7701-139756345206512"
Sep 08 20:44:44.074405 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.074362    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6277531-b656-4e8e-befa-70935d02010d" containerName="test-curl-7701-139756345206512"
Sep 08 20:44:44.074438 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.074429    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6277531-b656-4e8e-befa-70935d02010d" containerName="test-curl-7701-139756345206512"
Sep 08 20:44:44.077664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.077645    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:44.084181 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.084148    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:44.162770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.162725    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q859n\" (UniqueName: \"kubernetes.io/projected/b08e73b7-66ce-455e-aa1e-34e47d3ea4b6-kube-api-access-q859n\") pod \"test-curl-7701-139756345207088\" (UID: \"b08e73b7-66ce-455e-aa1e-34e47d3ea4b6\") " pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:44.263683 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.263649    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q859n\" (UniqueName: \"kubernetes.io/projected/b08e73b7-66ce-455e-aa1e-34e47d3ea4b6-kube-api-access-q859n\") pod \"test-curl-7701-139756345207088\" (UID: \"b08e73b7-66ce-455e-aa1e-34e47d3ea4b6\") " pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:44.272636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.272602    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q859n\" (UniqueName: \"kubernetes.io/projected/b08e73b7-66ce-455e-aa1e-34e47d3ea4b6-kube-api-access-q859n\") pod \"test-curl-7701-139756345207088\" (UID: \"b08e73b7-66ce-455e-aa1e-34e47d3ea4b6\") " pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:44.372828 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.372744    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6277531-b656-4e8e-befa-70935d02010d" path="/var/lib/kubelet/pods/e6277531-b656-4e8e-befa-70935d02010d/volumes"
Sep 08 20:44:44.389384 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.389356    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:44.666237 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.666208    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:44.666761 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:44.666737    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb08e73b7_66ce_455e_aa1e_34e47d3ea4b6.slice/crio-4038ff2b6bf7a8c36ba8f9b9dc0c438703212f6230644d3779f79a0ac36048f3 WatchSource:0}: Error finding container 4038ff2b6bf7a8c36ba8f9b9dc0c438703212f6230644d3779f79a0ac36048f3: Status 404 returned error can't find the container with id 4038ff2b6bf7a8c36ba8f9b9dc0c438703212f6230644d3779f79a0ac36048f3
Sep 08 20:44:44.773209 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.773170    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-139756345207088" event={"ID":"b08e73b7-66ce-455e-aa1e-34e47d3ea4b6","Type":"ContainerStarted","Data":"4038ff2b6bf7a8c36ba8f9b9dc0c438703212f6230644d3779f79a0ac36048f3"}
Sep 08 20:44:44.774625 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.774605    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345206512"
Sep 08 20:44:44.774764 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:44.774612    2561 scope.go:117] "RemoveContainer" containerID="31ac6290157dc0d267912d28759630fab8c6256a7b46e33c2075e9fa1e7cd452"
Sep 08 20:44:45.783678 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:45.783643    2561 generic.go:356] "Generic (PLEG): container finished" podID="b08e73b7-66ce-455e-aa1e-34e47d3ea4b6" containerID="c8c33326872e0b97f29028b13628f4b1f471ef62a8c90ba1adc0d8d43ab9a2a1" exitCode=0
Sep 08 20:44:45.784131 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:45.783721    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-139756345207088" event={"ID":"b08e73b7-66ce-455e-aa1e-34e47d3ea4b6","Type":"ContainerDied","Data":"c8c33326872e0b97f29028b13628f4b1f471ef62a8c90ba1adc0d8d43ab9a2a1"}
Sep 08 20:44:46.939636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:46.939606    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:46.960511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:46.960469    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7701-139756345207088_b08e73b7-66ce-455e-aa1e-34e47d3ea4b6/test-curl-7701-139756345207088/0.log"
Sep 08 20:44:46.984910 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:46.984866    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:46.990076 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:46.990028    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q859n\" (UniqueName: \"kubernetes.io/projected/b08e73b7-66ce-455e-aa1e-34e47d3ea4b6-kube-api-access-q859n\") pod \"b08e73b7-66ce-455e-aa1e-34e47d3ea4b6\" (UID: \"b08e73b7-66ce-455e-aa1e-34e47d3ea4b6\") "
Sep 08 20:44:46.990713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:46.990689    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:46.992373 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:46.992347    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b08e73b7-66ce-455e-aa1e-34e47d3ea4b6-kube-api-access-q859n" (OuterVolumeSpecName: "kube-api-access-q859n") pod "b08e73b7-66ce-455e-aa1e-34e47d3ea4b6" (UID: "b08e73b7-66ce-455e-aa1e-34e47d3ea4b6"). InnerVolumeSpecName "kube-api-access-q859n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:44:47.091979 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.091834    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q859n\" (UniqueName: \"kubernetes.io/projected/b08e73b7-66ce-455e-aa1e-34e47d3ea4b6-kube-api-access-q859n\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:44:47.106564 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.106529    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:47.107042 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.107028    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b08e73b7-66ce-455e-aa1e-34e47d3ea4b6" containerName="test-curl-7701-139756345207088"
Sep 08 20:44:47.107099 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.107045    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b08e73b7-66ce-455e-aa1e-34e47d3ea4b6" containerName="test-curl-7701-139756345207088"
Sep 08 20:44:47.107135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.107126    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b08e73b7-66ce-455e-aa1e-34e47d3ea4b6" containerName="test-curl-7701-139756345207088"
Sep 08 20:44:47.110285 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.110266    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:47.117357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.117328    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:47.193137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.193093    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf5cc\" (UniqueName: \"kubernetes.io/projected/27988c46-c9c1-4742-b2e2-a99a15192bb5-kube-api-access-nf5cc\") pod \"test-curl-7701-139756345207088\" (UID: \"27988c46-c9c1-4742-b2e2-a99a15192bb5\") " pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:47.294097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.294051    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf5cc\" (UniqueName: \"kubernetes.io/projected/27988c46-c9c1-4742-b2e2-a99a15192bb5-kube-api-access-nf5cc\") pod \"test-curl-7701-139756345207088\" (UID: \"27988c46-c9c1-4742-b2e2-a99a15192bb5\") " pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:47.303559 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.303526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf5cc\" (UniqueName: \"kubernetes.io/projected/27988c46-c9c1-4742-b2e2-a99a15192bb5-kube-api-access-nf5cc\") pod \"test-curl-7701-139756345207088\" (UID: \"27988c46-c9c1-4742-b2e2-a99a15192bb5\") " pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:47.421468 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.421431    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:47.696498 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.696471    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:47.697601 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:47.697532    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27988c46_c9c1_4742_b2e2_a99a15192bb5.slice/crio-5140041dd47a92ed9286d45b2c54bfb7b8dedf7b42b889135ff24b65d4053f5d WatchSource:0}: Error finding container 5140041dd47a92ed9286d45b2c54bfb7b8dedf7b42b889135ff24b65d4053f5d: Status 404 returned error can't find the container with id 5140041dd47a92ed9286d45b2c54bfb7b8dedf7b42b889135ff24b65d4053f5d
Sep 08 20:44:47.798884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.798858    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:47.798884 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.798890    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4038ff2b6bf7a8c36ba8f9b9dc0c438703212f6230644d3779f79a0ac36048f3"
Sep 08 20:44:47.800325 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.800297    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-139756345207088" event={"ID":"27988c46-c9c1-4742-b2e2-a99a15192bb5","Type":"ContainerStarted","Data":"5140041dd47a92ed9286d45b2c54bfb7b8dedf7b42b889135ff24b65d4053f5d"}
Sep 08 20:44:47.811469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:47.811440    2561 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7701-139756345207088" oldPodUID="b08e73b7-66ce-455e-aa1e-34e47d3ea4b6" podUID="27988c46-c9c1-4742-b2e2-a99a15192bb5"
Sep 08 20:44:48.366051 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:48.366019    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b08e73b7-66ce-455e-aa1e-34e47d3ea4b6" path="/var/lib/kubelet/pods/b08e73b7-66ce-455e-aa1e-34e47d3ea4b6/volumes"
Sep 08 20:44:48.808749 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:48.808707    2561 generic.go:356] "Generic (PLEG): container finished" podID="27988c46-c9c1-4742-b2e2-a99a15192bb5" containerID="182457d456125ca77a3f5ae58054377ec4fb8f0407fd313250b135568a753223" exitCode=0
Sep 08 20:44:48.808968 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:48.808806    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-139756345207088" event={"ID":"27988c46-c9c1-4742-b2e2-a99a15192bb5","Type":"ContainerDied","Data":"182457d456125ca77a3f5ae58054377ec4fb8f0407fd313250b135568a753223"}
Sep 08 20:44:49.975574 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:49.975548    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:49.993649 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:49.993599    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7701-139756345207088_27988c46-c9c1-4742-b2e2-a99a15192bb5/test-curl-7701-139756345207088/0.log"
Sep 08 20:44:50.016334 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.016296    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:50.019659 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.019630    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7701-139756345207088"]
Sep 08 20:44:50.021988 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.021968    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf5cc\" (UniqueName: \"kubernetes.io/projected/27988c46-c9c1-4742-b2e2-a99a15192bb5-kube-api-access-nf5cc\") pod \"27988c46-c9c1-4742-b2e2-a99a15192bb5\" (UID: \"27988c46-c9c1-4742-b2e2-a99a15192bb5\") "
Sep 08 20:44:50.024145 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.024116    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27988c46-c9c1-4742-b2e2-a99a15192bb5-kube-api-access-nf5cc" (OuterVolumeSpecName: "kube-api-access-nf5cc") pod "27988c46-c9c1-4742-b2e2-a99a15192bb5" (UID: "27988c46-c9c1-4742-b2e2-a99a15192bb5"). InnerVolumeSpecName "kube-api-access-nf5cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:44:50.123362 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.123252    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf5cc\" (UniqueName: \"kubernetes.io/projected/27988c46-c9c1-4742-b2e2-a99a15192bb5-kube-api-access-nf5cc\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:44:50.365056 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.365021    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27988c46-c9c1-4742-b2e2-a99a15192bb5" path="/var/lib/kubelet/pods/27988c46-c9c1-4742-b2e2-a99a15192bb5/volumes"
Sep 08 20:44:50.824984 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.824948    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-139756345207088"
Sep 08 20:44:50.825148 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.824948    2561 scope.go:117] "RemoveContainer" containerID="182457d456125ca77a3f5ae58054377ec4fb8f0407fd313250b135568a753223"
Sep 08 20:44:50.893182 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.893152    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"]
Sep 08 20:44:50.893733 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.893713    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27988c46-c9c1-4742-b2e2-a99a15192bb5" containerName="test-curl-7701-139756345207088"
Sep 08 20:44:50.893843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.893734    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="27988c46-c9c1-4742-b2e2-a99a15192bb5" containerName="test-curl-7701-139756345207088"
Sep 08 20:44:50.893933 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.893878    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="27988c46-c9c1-4742-b2e2-a99a15192bb5" containerName="test-curl-7701-139756345207088"
Sep 08 20:44:50.898978 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.898950    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.901970 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.901948    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e0dc0027-openshift-default-dockercfg-8hb8f\""
Sep 08 20:44:50.909737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.909713    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"]
Sep 08 20:44:50.935887 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.935847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-socket\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.936069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.935919    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-envoy\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.936069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.935959    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-podinfo\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.936069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.935978    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-data\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.936069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.936003    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcsq9\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-kube-api-access-rcsq9\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.936069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.936020    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-credential-socket\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.936308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.936107    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec19799-b71b-4419-a65d-b0ed01c22c37-istiod-ca-cert\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.936308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.936161    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-token\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:50.936308 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:50.936224    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-certs\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.036724 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.036684    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec19799-b71b-4419-a65d-b0ed01c22c37-istiod-ca-cert\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.036736    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-token\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.036782    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-certs\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.036914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-socket\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.036950    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-envoy\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.037032    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-podinfo\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.037065    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-data\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.037108    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcsq9\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-kube-api-access-rcsq9\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.037137    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-credential-socket\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037701 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.037555    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-data\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037701 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.037596    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-credential-socket\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037939 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.037602    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec19799-b71b-4419-a65d-b0ed01c22c37-istiod-ca-cert\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.037939 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.037840    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-socket\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.038299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.038276    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-certs\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.040075 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.040053    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-envoy\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.040294 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.040272    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-podinfo\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.047709 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.047689    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcsq9\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-kube-api-access-rcsq9\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.048047 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.048028    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-token\") pod \"e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") " pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.212277 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.212240    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:51.508868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.508839    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"]
Sep 08 20:44:51.509784 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:51.509759    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec19799_b71b_4419_a65d_b0ed01c22c37.slice/crio-1f8400e2a63bea4f0700b44af232a2e93e127a3752fc3513da742d1530d20208 WatchSource:0}: Error finding container 1f8400e2a63bea4f0700b44af232a2e93e127a3752fc3513da742d1530d20208: Status 404 returned error can't find the container with id 1f8400e2a63bea4f0700b44af232a2e93e127a3752fc3513da742d1530d20208
Sep 08 20:44:51.512271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.512230    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:44:51.512351 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.512329    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:44:51.512410 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.512388    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:44:51.836524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.836425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" event={"ID":"fec19799-b71b-4419-a65d-b0ed01c22c37","Type":"ContainerStarted","Data":"ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d"}
Sep 08 20:44:51.836524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.836483    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" event={"ID":"fec19799-b71b-4419-a65d-b0ed01c22c37","Type":"ContainerStarted","Data":"1f8400e2a63bea4f0700b44af232a2e93e127a3752fc3513da742d1530d20208"}
Sep 08 20:44:51.861036 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:51.860978    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podStartSLOduration=1.860959628 podStartE2EDuration="1.860959628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:51.85808609 +0000 UTC m=+1178.023256658" watchObservedRunningTime="2026-09-08 20:44:51.860959628 +0000 UTC m=+1178.026130201"
Sep 08 20:44:52.213210 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:52.213173    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:52.219044 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:52.219016    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:52.843666 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:52.843632    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:52.844775 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:52.844752    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:44:55.602566 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.602533    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"]
Sep 08 20:44:55.606454 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.606436    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:55.610787 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.610767    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e0dc0027-dockercfg-w2lcg\""
Sep 08 20:44:55.611047 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.611031    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e0dc0027\""
Sep 08 20:44:55.623462 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.623430    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"]
Sep 08 20:44:55.690313 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.690278    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6dbl\" (UniqueName: \"kubernetes.io/projected/8904a49f-3d18-4274-9748-0056f3aa3f7a-kube-api-access-j6dbl\") pod \"payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp\" (UID: \"8904a49f-3d18-4274-9748-0056f3aa3f7a\") " pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:55.690524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.690331    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8904a49f-3d18-4274-9748-0056f3aa3f7a-plugins-config-volume\") pod \"payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp\" (UID: \"8904a49f-3d18-4274-9748-0056f3aa3f7a\") " pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:55.791443 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.791395    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6dbl\" (UniqueName: \"kubernetes.io/projected/8904a49f-3d18-4274-9748-0056f3aa3f7a-kube-api-access-j6dbl\") pod \"payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp\" (UID: \"8904a49f-3d18-4274-9748-0056f3aa3f7a\") " pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:55.791646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.791478    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8904a49f-3d18-4274-9748-0056f3aa3f7a-plugins-config-volume\") pod \"payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp\" (UID: \"8904a49f-3d18-4274-9748-0056f3aa3f7a\") " pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:55.792276 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.792248    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8904a49f-3d18-4274-9748-0056f3aa3f7a-plugins-config-volume\") pod \"payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp\" (UID: \"8904a49f-3d18-4274-9748-0056f3aa3f7a\") " pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:55.800737 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.800715    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6dbl\" (UniqueName: \"kubernetes.io/projected/8904a49f-3d18-4274-9748-0056f3aa3f7a-kube-api-access-j6dbl\") pod \"payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp\" (UID: \"8904a49f-3d18-4274-9748-0056f3aa3f7a\") " pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:55.918133 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:55.918099    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:56.197890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:56.197860    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"]
Sep 08 20:44:56.198507 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:44:56.198486    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8904a49f_3d18_4274_9748_0056f3aa3f7a.slice/crio-ac04ab75d30d29d48412df73f4488ceccf64a960a99ceb821f838991f81703f1 WatchSource:0}: Error finding container ac04ab75d30d29d48412df73f4488ceccf64a960a99ceb821f838991f81703f1: Status 404 returned error can't find the container with id ac04ab75d30d29d48412df73f4488ceccf64a960a99ceb821f838991f81703f1
Sep 08 20:44:56.875207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:56.875160    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp" event={"ID":"8904a49f-3d18-4274-9748-0056f3aa3f7a","Type":"ContainerStarted","Data":"67bc87743779aff037d3a84d805eb2e363f52d2311ede84d3df19f6d2f7a27c7"}
Sep 08 20:44:56.875703 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:56.875228    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp" event={"ID":"8904a49f-3d18-4274-9748-0056f3aa3f7a","Type":"ContainerStarted","Data":"ac04ab75d30d29d48412df73f4488ceccf64a960a99ceb821f838991f81703f1"}
Sep 08 20:44:56.875703 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:56.875258    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:44:56.893057 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:44:56.892992    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp" podStartSLOduration=1.89297009 podStartE2EDuration="1.89297009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:56.89124419 +0000 UTC m=+1183.056414776" watchObservedRunningTime="2026-09-08 20:44:56.89297009 +0000 UTC m=+1183.058140673"
Sep 08 20:45:00.140630 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.140592    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"]
Sep 08 20:45:00.144099 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.144077    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"]
Sep 08 20:45:00.144303 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.144282    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"
Sep 08 20:45:00.146906 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.146864    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:45:00.147789 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.147774    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km"]
Sep 08 20:45:00.147951 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.147931    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"
Sep 08 20:45:00.151481 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.151456    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"]
Sep 08 20:45:00.151729 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.151644    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km"
Sep 08 20:45:00.156510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.156473    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"]
Sep 08 20:45:00.156657 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.156603    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"
Sep 08 20:45:00.158876 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.158843    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"]
Sep 08 20:45:00.159418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.159399    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"
Sep 08 20:45:00.166571 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.164416    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"]
Sep 08 20:45:00.169215 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.169184    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km"]
Sep 08 20:45:00.178043 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.178017    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"
Sep 08 20:45:00.226332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.226291    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km"
Sep 08 20:45:00.236301 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.236275    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"
Sep 08 20:45:00.477428 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.477393    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"]
Sep 08 20:45:00.478977 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:45:00.478947    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d10b58_7790_424e_b1e3_21ec85f21b0b.slice/crio-86ef703ed78993b932e64a2f2f2059c8964678568f524fad1668dd7327d77698 WatchSource:0}: Error finding container 86ef703ed78993b932e64a2f2f2059c8964678568f524fad1668dd7327d77698: Status 404 returned error can't find the container with id 86ef703ed78993b932e64a2f2f2059c8964678568f524fad1668dd7327d77698
Sep 08 20:45:00.499080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.499054    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"]
Sep 08 20:45:00.500958 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:45:00.500930    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9feedae1_c30d_4c00_bcff_e219a526768a.slice/crio-7af28876f267d5af94838e10efef5a34ec82e65a7608904fe7774ecaf8ba9114 WatchSource:0}: Error finding container 7af28876f267d5af94838e10efef5a34ec82e65a7608904fe7774ecaf8ba9114: Status 404 returned error can't find the container with id 7af28876f267d5af94838e10efef5a34ec82e65a7608904fe7774ecaf8ba9114
Sep 08 20:45:00.549584 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.549551    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km"]
Sep 08 20:45:00.553500 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.553467    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"]
Sep 08 20:45:00.635183 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:45:00.635014    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec337b6_07d9_46c9_af42_e90d346ad8ef.slice/crio-f81c4998be0314c38d1fc767783da67be108c4fe44f58719e8b23a856b1ed2a1 WatchSource:0}: Error finding container f81c4998be0314c38d1fc767783da67be108c4fe44f58719e8b23a856b1ed2a1: Status 404 returned error can't find the container with id f81c4998be0314c38d1fc767783da67be108c4fe44f58719e8b23a856b1ed2a1
Sep 08 20:45:00.906187 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.906149    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d" event={"ID":"9feedae1-c30d-4c00-bcff-e219a526768a","Type":"ContainerStarted","Data":"3dff9967405d41192ed6b1d8215faf0376e969ff0de120bdebfda8f733baee08"}
Sep 08 20:45:00.906400 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.906217    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d" event={"ID":"9feedae1-c30d-4c00-bcff-e219a526768a","Type":"ContainerStarted","Data":"7af28876f267d5af94838e10efef5a34ec82e65a7608904fe7774ecaf8ba9114"}
Sep 08 20:45:00.907579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.907545    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km" event={"ID":"5ec337b6-07d9-46c9-af42-e90d346ad8ef","Type":"ContainerStarted","Data":"f81c4998be0314c38d1fc767783da67be108c4fe44f58719e8b23a856b1ed2a1"}
Sep 08 20:45:00.909231 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.909195    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5" event={"ID":"e9d10b58-7790-424e-b1e3-21ec85f21b0b","Type":"ContainerStarted","Data":"5229ba28ce6b8ec00cbef5d4cf8c813a151e91a8072bbed6692610de879af257"}
Sep 08 20:45:00.909367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.909254    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5" event={"ID":"e9d10b58-7790-424e-b1e3-21ec85f21b0b","Type":"ContainerStarted","Data":"86ef703ed78993b932e64a2f2f2059c8964678568f524fad1668dd7327d77698"}
Sep 08 20:45:00.910582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.910559    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" event={"ID":"ee94d222-1a7a-49b7-8ea2-749d2ab4db4b","Type":"ContainerStarted","Data":"b229df2191ffd9b03a86f1305b0654cf6a0c91b0572a56452aabd3254a7445d9"}
Sep 08 20:45:00.922130 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.922075    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d" podStartSLOduration=0.922056633 podStartE2EDuration="922.056633ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:45:00.920015703 +0000 UTC m=+1187.085186280" watchObservedRunningTime="2026-09-08 20:45:00.922056633 +0000 UTC m=+1187.087227268"
Sep 08 20:45:00.937535 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:00.937465    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5" podStartSLOduration=0.937443057 podStartE2EDuration="937.443057ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:45:00.934444805 +0000 UTC m=+1187.099615382" watchObservedRunningTime="2026-09-08 20:45:00.937443057 +0000 UTC m=+1187.102613634"
Sep 08 20:45:01.919796 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.919760    2561 generic.go:356] "Generic (PLEG): container finished" podID="9feedae1-c30d-4c00-bcff-e219a526768a" containerID="3dff9967405d41192ed6b1d8215faf0376e969ff0de120bdebfda8f733baee08" exitCode=0
Sep 08 20:45:01.920231 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.919848    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d" event={"ID":"9feedae1-c30d-4c00-bcff-e219a526768a","Type":"ContainerDied","Data":"3dff9967405d41192ed6b1d8215faf0376e969ff0de120bdebfda8f733baee08"}
Sep 08 20:45:01.921568 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.921539    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km" event={"ID":"5ec337b6-07d9-46c9-af42-e90d346ad8ef","Type":"ContainerStarted","Data":"ebb76e009309a03fcebabe79e560a49cdbb29d755c65f94fadf92d51940f88c3"}
Sep 08 20:45:01.923116 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.923092    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9d10b58-7790-424e-b1e3-21ec85f21b0b" containerID="5229ba28ce6b8ec00cbef5d4cf8c813a151e91a8072bbed6692610de879af257" exitCode=0
Sep 08 20:45:01.923241 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.923121    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5" event={"ID":"e9d10b58-7790-424e-b1e3-21ec85f21b0b","Type":"ContainerDied","Data":"5229ba28ce6b8ec00cbef5d4cf8c813a151e91a8072bbed6692610de879af257"}
Sep 08 20:45:01.924841 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.924821    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerID="97d843fdd33cf6e2c50d6d3948e4fbcb40647d2df6480afceb70f7edc6f86c36" exitCode=7
Sep 08 20:45:01.924955 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.924889    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" event={"ID":"ee94d222-1a7a-49b7-8ea2-749d2ab4db4b","Type":"ContainerDied","Data":"97d843fdd33cf6e2c50d6d3948e4fbcb40647d2df6480afceb70f7edc6f86c36"}
Sep 08 20:45:01.925049 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.925037    2561 scope.go:117] "RemoveContainer" containerID="97d843fdd33cf6e2c50d6d3948e4fbcb40647d2df6480afceb70f7edc6f86c36"
Sep 08 20:45:01.951441 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:01.951367    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km" podStartSLOduration=1.9513478119999998 podStartE2EDuration="1.951347812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:45:01.948803643 +0000 UTC m=+1188.113974234" watchObservedRunningTime="2026-09-08 20:45:01.951347812 +0000 UTC m=+1188.116518388"
Sep 08 20:45:02.934679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:02.934641    2561 generic.go:356] "Generic (PLEG): container finished" podID="5ec337b6-07d9-46c9-af42-e90d346ad8ef" containerID="ebb76e009309a03fcebabe79e560a49cdbb29d755c65f94fadf92d51940f88c3" exitCode=0
Sep 08 20:45:02.935183 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:02.934744    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km" event={"ID":"5ec337b6-07d9-46c9-af42-e90d346ad8ef","Type":"ContainerDied","Data":"ebb76e009309a03fcebabe79e560a49cdbb29d755c65f94fadf92d51940f88c3"}
Sep 08 20:45:02.937391 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:02.937355    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerID="63de587f5e85de2185e834c633605a80376d7c65cc8b7f822ef3e788a71ee80e" exitCode=7
Sep 08 20:45:02.937589 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:02.937465    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" event={"ID":"ee94d222-1a7a-49b7-8ea2-749d2ab4db4b","Type":"ContainerDied","Data":"63de587f5e85de2185e834c633605a80376d7c65cc8b7f822ef3e788a71ee80e"}
Sep 08 20:45:02.937589 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:02.937518    2561 scope.go:117] "RemoveContainer" containerID="97d843fdd33cf6e2c50d6d3948e4fbcb40647d2df6480afceb70f7edc6f86c36"
Sep 08 20:45:02.937793 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:02.937777    2561 scope.go:117] "RemoveContainer" containerID="63de587f5e85de2185e834c633605a80376d7c65cc8b7f822ef3e788a71ee80e"
Sep 08 20:45:02.938298 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:45:02.938123    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv_odh-ai-gateway-infra(ee94d222-1a7a-49b7-8ea2-749d2ab4db4b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b"
Sep 08 20:45:03.114139 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.114112    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"
Sep 08 20:45:03.157096 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.157067    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"
Sep 08 20:45:03.949697 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.949664    2561 scope.go:117] "RemoveContainer" containerID="63de587f5e85de2185e834c633605a80376d7c65cc8b7f822ef3e788a71ee80e"
Sep 08 20:45:03.950111 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:45:03.949995    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv_odh-ai-gateway-infra(ee94d222-1a7a-49b7-8ea2-749d2ab4db4b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b"
Sep 08 20:45:03.950840 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.950820    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"
Sep 08 20:45:03.950994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.950849    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d" event={"ID":"9feedae1-c30d-4c00-bcff-e219a526768a","Type":"ContainerDied","Data":"7af28876f267d5af94838e10efef5a34ec82e65a7608904fe7774ecaf8ba9114"}
Sep 08 20:45:03.950994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.950890    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7af28876f267d5af94838e10efef5a34ec82e65a7608904fe7774ecaf8ba9114"
Sep 08 20:45:03.952321 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.952298    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"
Sep 08 20:45:03.952418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.952329    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5" event={"ID":"e9d10b58-7790-424e-b1e3-21ec85f21b0b","Type":"ContainerDied","Data":"86ef703ed78993b932e64a2f2f2059c8964678568f524fad1668dd7327d77698"}
Sep 08 20:45:03.952418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:03.952378    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86ef703ed78993b932e64a2f2f2059c8964678568f524fad1668dd7327d77698"
Sep 08 20:45:04.093251 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:04.093225    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km"
Sep 08 20:45:04.960257 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:04.960227    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km"
Sep 08 20:45:04.960646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:04.960231    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815005-jd4km" event={"ID":"5ec337b6-07d9-46c9-af42-e90d346ad8ef","Type":"ContainerDied","Data":"f81c4998be0314c38d1fc767783da67be108c4fe44f58719e8b23a856b1ed2a1"}
Sep 08 20:45:04.960646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:04.960347    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f81c4998be0314c38d1fc767783da67be108c4fe44f58719e8b23a856b1ed2a1"
Sep 08 20:45:07.885073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:07.885042    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:45:14.452612 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:14.452582    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:45:14.455484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:14.455456    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:45:15.037815 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:15.037767    2561 generic.go:356] "Generic (PLEG): container finished" podID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerID="ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d" exitCode=0
Sep 08 20:45:15.038011 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:15.037840    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" event={"ID":"fec19799-b71b-4419-a65d-b0ed01c22c37","Type":"ContainerDied","Data":"ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d"}
Sep 08 20:45:15.038373 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:15.038355    2561 scope.go:117] "RemoveContainer" containerID="ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d"
Sep 08 20:45:15.039684 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:15.039639    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:45:15.039806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:15.039724    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:45:15.039806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:15.039757    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:45:16.053353 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:16.053311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" event={"ID":"fec19799-b71b-4419-a65d-b0ed01c22c37","Type":"ContainerStarted","Data":"1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f"}
Sep 08 20:45:16.212808 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:16.212769    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:45:16.213041 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:16.213015    2561 patch_prober.go:28] interesting pod/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 08 20:45:16.213129 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:16.213079    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 08 20:45:17.213368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:17.213330    2561 patch_prober.go:28] interesting pod/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 08 20:45:17.213777 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:17.213399    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 08 20:45:18.213552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:18.213513    2561 patch_prober.go:28] interesting pod/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 08 20:45:18.213960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:18.213571    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 08 20:45:19.213475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:19.213437    2561 patch_prober.go:28] interesting pod/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 08 20:45:19.213632 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:19.213499    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 08 20:45:19.358288 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:19.358256    2561 scope.go:117] "RemoveContainer" containerID="63de587f5e85de2185e834c633605a80376d7c65cc8b7f822ef3e788a71ee80e"
Sep 08 20:45:20.087180 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:20.087145    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" event={"ID":"ee94d222-1a7a-49b7-8ea2-749d2ab4db4b","Type":"ContainerStarted","Data":"503cf50a63df6e1d9078f22819acf21dff768079cd3a48d483b382b367ffeb38"}
Sep 08 20:45:20.105222 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:20.105164    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" podStartSLOduration=20.105146653 podStartE2EDuration="20.105146653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:45:20.102261404 +0000 UTC m=+1206.267431981" watchObservedRunningTime="2026-09-08 20:45:20.105146653 +0000 UTC m=+1206.270317263"
Sep 08 20:45:20.213079 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:20.213043    2561 patch_prober.go:28] interesting pod/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 08 20:45:20.213261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:20.213100    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 08 20:45:21.096192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:21.096098    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerID="503cf50a63df6e1d9078f22819acf21dff768079cd3a48d483b382b367ffeb38" exitCode=0
Sep 08 20:45:21.096192 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:21.096174    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" event={"ID":"ee94d222-1a7a-49b7-8ea2-749d2ab4db4b","Type":"ContainerDied","Data":"503cf50a63df6e1d9078f22819acf21dff768079cd3a48d483b382b367ffeb38"}
Sep 08 20:45:21.096716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:21.096222    2561 scope.go:117] "RemoveContainer" containerID="63de587f5e85de2185e834c633605a80376d7c65cc8b7f822ef3e788a71ee80e"
Sep 08 20:45:21.114039 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:21.114008    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"]
Sep 08 20:45:21.213694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:21.213657    2561 patch_prober.go:28] interesting pod/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 08 20:45:21.213868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:21.213732    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 08 20:45:22.216984 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:22.216953    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:45:22.217375 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:22.217221    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:45:22.218065 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:22.218044    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:45:22.260994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:22.260964    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"
Sep 08 20:45:23.112723 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:23.112687    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"
Sep 08 20:45:23.112955 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:23.112725    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv" event={"ID":"ee94d222-1a7a-49b7-8ea2-749d2ab4db4b","Type":"ContainerDied","Data":"b229df2191ffd9b03a86f1305b0654cf6a0c91b0572a56452aabd3254a7445d9"}
Sep 08 20:45:23.112955 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:23.112785    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b229df2191ffd9b03a86f1305b0654cf6a0c91b0572a56452aabd3254a7445d9"
Sep 08 20:45:23.144241 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:23.144206    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"]
Sep 08 20:45:23.149144 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:23.149112    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e0dc0027-29815005-xvlsv"]
Sep 08 20:45:24.364128 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:45:24.364098    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" path="/var/lib/kubelet/pods/ee94d222-1a7a-49b7-8ea2-749d2ab4db4b/volumes"
Sep 08 20:46:13.848654 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.848566    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"]
Sep 08 20:46:13.849173 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849155    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:46:13.849227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849175    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:46:13.849227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849185    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9feedae1-c30d-4c00-bcff-e219a526768a" containerName="cleanup"
Sep 08 20:46:13.849227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849190    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9feedae1-c30d-4c00-bcff-e219a526768a" containerName="cleanup"
Sep 08 20:46:13.849227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849213    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d10b58-7790-424e-b1e3-21ec85f21b0b" containerName="cleanup"
Sep 08 20:46:13.849227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849219    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d10b58-7790-424e-b1e3-21ec85f21b0b" containerName="cleanup"
Sep 08 20:46:13.849227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849226    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ec337b6-07d9-46c9-af42-e90d346ad8ef" containerName="cleanup"
Sep 08 20:46:13.849415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849231    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ec337b6-07d9-46c9-af42-e90d346ad8ef" containerName="cleanup"
Sep 08 20:46:13.849415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849239    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:46:13.849415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849243    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:46:13.849415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849335    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:46:13.849415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849343    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ec337b6-07d9-46c9-af42-e90d346ad8ef" containerName="cleanup"
Sep 08 20:46:13.849415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849352    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9feedae1-c30d-4c00-bcff-e219a526768a" containerName="cleanup"
Sep 08 20:46:13.849415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849360    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:46:13.849415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.849365    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d10b58-7790-424e-b1e3-21ec85f21b0b" containerName="cleanup"
Sep 08 20:46:13.852628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.852610    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"
Sep 08 20:46:13.856424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.856395    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:46:13.859424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.859399    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"]
Sep 08 20:46:13.913163 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:13.913128    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426064bc-8910-44a1-bb47-6afc78756611-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx\" (UID: \"426064bc-8910-44a1-bb47-6afc78756611\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"
Sep 08 20:46:14.014432 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:14.014386    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426064bc-8910-44a1-bb47-6afc78756611-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx\" (UID: \"426064bc-8910-44a1-bb47-6afc78756611\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"
Sep 08 20:46:14.015322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:14.015282    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426064bc-8910-44a1-bb47-6afc78756611-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx\" (UID: \"426064bc-8910-44a1-bb47-6afc78756611\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"
Sep 08 20:46:14.167303 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:14.167266    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"
Sep 08 20:46:14.443180 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:14.443151    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"]
Sep 08 20:46:14.445488 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:46:14.445447    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426064bc_8910_44a1_bb47_6afc78756611.slice/crio-6f23d11b4ba02616427c1dff45c76a84d4b103be074c9a7c49a769ca571bc647 WatchSource:0}: Error finding container 6f23d11b4ba02616427c1dff45c76a84d4b103be074c9a7c49a769ca571bc647: Status 404 returned error can't find the container with id 6f23d11b4ba02616427c1dff45c76a84d4b103be074c9a7c49a769ca571bc647
Sep 08 20:46:14.468568 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:14.468526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx" event={"ID":"426064bc-8910-44a1-bb47-6afc78756611","Type":"ContainerStarted","Data":"6f23d11b4ba02616427c1dff45c76a84d4b103be074c9a7c49a769ca571bc647"}
Sep 08 20:46:15.477239 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:15.477203    2561 generic.go:356] "Generic (PLEG): container finished" podID="426064bc-8910-44a1-bb47-6afc78756611" containerID="aeb3cb7887fd2994d7270448afff52ad486f7f6df15e4f5b25df9a33ebff867d" exitCode=0
Sep 08 20:46:15.477649 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:15.477265    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx" event={"ID":"426064bc-8910-44a1-bb47-6afc78756611","Type":"ContainerDied","Data":"aeb3cb7887fd2994d7270448afff52ad486f7f6df15e4f5b25df9a33ebff867d"}
Sep 08 20:46:16.637440 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:16.637414    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"
Sep 08 20:46:16.742266 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:16.742160    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426064bc-8910-44a1-bb47-6afc78756611-maas-api-service-ca\") pod \"426064bc-8910-44a1-bb47-6afc78756611\" (UID: \"426064bc-8910-44a1-bb47-6afc78756611\") "
Sep 08 20:46:16.742535 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:16.742510    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/426064bc-8910-44a1-bb47-6afc78756611-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "426064bc-8910-44a1-bb47-6afc78756611" (UID: "426064bc-8910-44a1-bb47-6afc78756611"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:46:16.843497 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:16.843455    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/426064bc-8910-44a1-bb47-6afc78756611-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:46:17.493668 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:17.493637    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"
Sep 08 20:46:17.493668 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:17.493663    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx" event={"ID":"426064bc-8910-44a1-bb47-6afc78756611","Type":"ContainerDied","Data":"6f23d11b4ba02616427c1dff45c76a84d4b103be074c9a7c49a769ca571bc647"}
Sep 08 20:46:17.493922 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:17.493710    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f23d11b4ba02616427c1dff45c76a84d4b103be074c9a7c49a769ca571bc647"
Sep 08 20:46:29.367686 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.367649    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"]
Sep 08 20:46:29.368242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.367864    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp" podUID="8904a49f-3d18-4274-9748-0056f3aa3f7a" containerName="payload-processing" containerID="cri-o://67bc87743779aff037d3a84d805eb2e363f52d2311ede84d3df19f6d2f7a27c7" gracePeriod=30
Sep 08 20:46:29.551887 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.551841    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"]
Sep 08 20:46:29.554978 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.554953    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e0dc0027-hrkxx"]
Sep 08 20:46:29.583526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.583490    2561 generic.go:356] "Generic (PLEG): container finished" podID="8904a49f-3d18-4274-9748-0056f3aa3f7a" containerID="67bc87743779aff037d3a84d805eb2e363f52d2311ede84d3df19f6d2f7a27c7" exitCode=0
Sep 08 20:46:29.583681 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.583561    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp" event={"ID":"8904a49f-3d18-4274-9748-0056f3aa3f7a","Type":"ContainerDied","Data":"67bc87743779aff037d3a84d805eb2e363f52d2311ede84d3df19f6d2f7a27c7"}
Sep 08 20:46:29.653992 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.653962    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:46:29.678494 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.678461    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8904a49f-3d18-4274-9748-0056f3aa3f7a-plugins-config-volume\") pod \"8904a49f-3d18-4274-9748-0056f3aa3f7a\" (UID: \"8904a49f-3d18-4274-9748-0056f3aa3f7a\") "
Sep 08 20:46:29.678679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.678588    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6dbl\" (UniqueName: \"kubernetes.io/projected/8904a49f-3d18-4274-9748-0056f3aa3f7a-kube-api-access-j6dbl\") pod \"8904a49f-3d18-4274-9748-0056f3aa3f7a\" (UID: \"8904a49f-3d18-4274-9748-0056f3aa3f7a\") "
Sep 08 20:46:29.678888 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.678861    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8904a49f-3d18-4274-9748-0056f3aa3f7a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8904a49f-3d18-4274-9748-0056f3aa3f7a" (UID: "8904a49f-3d18-4274-9748-0056f3aa3f7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:46:29.680676 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.680649    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8904a49f-3d18-4274-9748-0056f3aa3f7a-kube-api-access-j6dbl" (OuterVolumeSpecName: "kube-api-access-j6dbl") pod "8904a49f-3d18-4274-9748-0056f3aa3f7a" (UID: "8904a49f-3d18-4274-9748-0056f3aa3f7a"). InnerVolumeSpecName "kube-api-access-j6dbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:46:29.780394 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.780340    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8904a49f-3d18-4274-9748-0056f3aa3f7a-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:46:29.780394 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:29.780380    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6dbl\" (UniqueName: \"kubernetes.io/projected/8904a49f-3d18-4274-9748-0056f3aa3f7a-kube-api-access-j6dbl\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:46:30.369475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:30.369436    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="426064bc-8910-44a1-bb47-6afc78756611" path="/var/lib/kubelet/pods/426064bc-8910-44a1-bb47-6afc78756611/volumes"
Sep 08 20:46:30.592277 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:30.592248    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"
Sep 08 20:46:30.592469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:30.592292    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp" event={"ID":"8904a49f-3d18-4274-9748-0056f3aa3f7a","Type":"ContainerDied","Data":"ac04ab75d30d29d48412df73f4488ceccf64a960a99ceb821f838991f81703f1"}
Sep 08 20:46:30.592469 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:30.592359    2561 scope.go:117] "RemoveContainer" containerID="67bc87743779aff037d3a84d805eb2e363f52d2311ede84d3df19f6d2f7a27c7"
Sep 08 20:46:30.610714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:30.610677    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"]
Sep 08 20:46:30.614238 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:30.614201    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e0dc0027-7749d48469-bdssp"]
Sep 08 20:46:32.364579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:46:32.364542    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8904a49f-3d18-4274-9748-0056f3aa3f7a" path="/var/lib/kubelet/pods/8904a49f-3d18-4274-9748-0056f3aa3f7a/volumes"
Sep 08 20:47:14.642831 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:14.642790    2561 scope.go:117] "RemoveContainer" containerID="da99f61f706b46a0eb6876eb4c5104af395b65f8329f32080c0addb1e6627d6a"
Sep 08 20:47:14.653681 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:14.653654    2561 scope.go:117] "RemoveContainer" containerID="9e729f9c57559e9ec426795e11da57c5704ad8122a902aeddb0f3ce625f70085"
Sep 08 20:47:14.663546 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:14.663517    2561 scope.go:117] "RemoveContainer" containerID="2590248b4b4fa280256ccdd2b124eb53cee613d57a0ec6ddba773fda616bbf83"
Sep 08 20:47:36.088872 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:36.088822    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"]
Sep 08 20:47:36.089446 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:36.089134    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy" containerID="cri-o://1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f" gracePeriod=30
Sep 08 20:47:37.220552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:37.220515    2561 patch_prober.go:28] interesting pod/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 20:47:37.221041 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:37.220586    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 20:47:41.360760 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.360698    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:47:41.419081 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419043    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-envoy\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419268 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419106    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-credential-socket\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419268 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419139    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-token\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419268 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419175    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-certs\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419268 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419264    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-data\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419303    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec19799-b71b-4419-a65d-b0ed01c22c37-istiod-ca-cert\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419348    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-podinfo\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419379    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:47:41.419489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419410    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcsq9\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-kube-api-access-rcsq9\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419445    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-socket\") pod \"fec19799-b71b-4419-a65d-b0ed01c22c37\" (UID: \"fec19799-b71b-4419-a65d-b0ed01c22c37\") "
Sep 08 20:47:41.419759 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419502    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:47:41.419759 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419511    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-data" (OuterVolumeSpecName: "istio-data") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:47:41.419865 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419760    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:47:41.419865 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419780    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fec19799-b71b-4419-a65d-b0ed01c22c37-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:47:41.420001 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419883    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:41.420001 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419927    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec19799-b71b-4419-a65d-b0ed01c22c37-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:41.420001 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419940    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:41.420001 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419953    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:41.420001 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.419968    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:41.421498 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.421462    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:47:41.421784 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.421763    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:47:41.421784 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.421773    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-token" (OuterVolumeSpecName: "istio-token") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:47:41.422001 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.421947    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-kube-api-access-rcsq9" (OuterVolumeSpecName: "kube-api-access-rcsq9") pod "fec19799-b71b-4419-a65d-b0ed01c22c37" (UID: "fec19799-b71b-4419-a65d-b0ed01c22c37"). InnerVolumeSpecName "kube-api-access-rcsq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:47:41.520583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.520542    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:41.520583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.520575    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcsq9\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-kube-api-access-rcsq9\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:41.520583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.520585    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:41.520583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:41.520594    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec19799-b71b-4419-a65d-b0ed01c22c37-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:42.111729 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.111688    2561 generic.go:356] "Generic (PLEG): container finished" podID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerID="1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f" exitCode=0
Sep 08 20:47:42.111926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.111768    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"
Sep 08 20:47:42.111926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.111772    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" event={"ID":"fec19799-b71b-4419-a65d-b0ed01c22c37","Type":"ContainerDied","Data":"1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f"}
Sep 08 20:47:42.111926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.111907    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9" event={"ID":"fec19799-b71b-4419-a65d-b0ed01c22c37","Type":"ContainerDied","Data":"1f8400e2a63bea4f0700b44af232a2e93e127a3752fc3513da742d1530d20208"}
Sep 08 20:47:42.111926 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.111919    2561 scope.go:117] "RemoveContainer" containerID="1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f"
Sep 08 20:47:42.122665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.122642    2561 scope.go:117] "RemoveContainer" containerID="ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d"
Sep 08 20:47:42.141092 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.141056    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"]
Sep 08 20:47:42.143886 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.143863    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e0dc0027-openshift-default-b856bdb68-8dpp9"]
Sep 08 20:47:42.147017 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.146996    2561 scope.go:117] "RemoveContainer" containerID="1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f"
Sep 08 20:47:42.147400 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:47:42.147379    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f\": container with ID starting with 1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f not found: ID does not exist" containerID="1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f"
Sep 08 20:47:42.147461 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.147410    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f"} err="failed to get container status \"1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f\": rpc error: code = NotFound desc = could not find container \"1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f\": container with ID starting with 1ad2d2c02e64dd231dab84447be2e417b6e226cb5bb9cc137a5d44421e91a44f not found: ID does not exist"
Sep 08 20:47:42.147461 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.147430    2561 scope.go:117] "RemoveContainer" containerID="ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d"
Sep 08 20:47:42.147759 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:47:42.147737    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d\": container with ID starting with ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d not found: ID does not exist" containerID="ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d"
Sep 08 20:47:42.147816 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.147767    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d"} err="failed to get container status \"ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d\": rpc error: code = NotFound desc = could not find container \"ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d\": container with ID starting with ea88e2e28b2fd8827044e07a24b9b3fbaf98217f949ce5adc340cc3420111a3d not found: ID does not exist"
Sep 08 20:47:42.367444 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:42.367347    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" path="/var/lib/kubelet/pods/fec19799-b71b-4419-a65d-b0ed01c22c37/volumes"
Sep 08 20:47:52.070205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070171    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7704-7369dc"]
Sep 08 20:47:52.070694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070679    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426064bc-8910-44a1-bb47-6afc78756611" containerName="revoke-keys"
Sep 08 20:47:52.070738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070695    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="426064bc-8910-44a1-bb47-6afc78756611" containerName="revoke-keys"
Sep 08 20:47:52.070738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070711    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:47:52.070738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070716    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:47:52.070738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070728    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8904a49f-3d18-4274-9748-0056f3aa3f7a" containerName="payload-processing"
Sep 08 20:47:52.070738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070734    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8904a49f-3d18-4274-9748-0056f3aa3f7a" containerName="payload-processing"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070746    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070751    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070767    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070773    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070836    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8904a49f-3d18-4274-9748-0056f3aa3f7a" containerName="payload-processing"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070844    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070852    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fec19799-b71b-4419-a65d-b0ed01c22c37" containerName="istio-proxy"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070858    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee94d222-1a7a-49b7-8ea2-749d2ab4db4b" containerName="cleanup"
Sep 08 20:47:52.070890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.070865    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="426064bc-8910-44a1-bb47-6afc78756611" containerName="revoke-keys"
Sep 08 20:47:52.074143 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.074127    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-7369dc"
Sep 08 20:47:52.076495 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.076465    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qqm77\""
Sep 08 20:47:52.081124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.081104    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7704-7369dc"]
Sep 08 20:47:52.144857 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.144824    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx5z5\" (UniqueName: \"kubernetes.io/projected/dde0d543-6b9c-4c39-b69d-1b9f341a518b-kube-api-access-mx5z5\") pod \"mt-curl-7704-7369dc\" (UID: \"dde0d543-6b9c-4c39-b69d-1b9f341a518b\") " pod="opendatahub/mt-curl-7704-7369dc"
Sep 08 20:47:52.245981 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.245846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx5z5\" (UniqueName: \"kubernetes.io/projected/dde0d543-6b9c-4c39-b69d-1b9f341a518b-kube-api-access-mx5z5\") pod \"mt-curl-7704-7369dc\" (UID: \"dde0d543-6b9c-4c39-b69d-1b9f341a518b\") " pod="opendatahub/mt-curl-7704-7369dc"
Sep 08 20:47:52.254384 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.254357    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx5z5\" (UniqueName: \"kubernetes.io/projected/dde0d543-6b9c-4c39-b69d-1b9f341a518b-kube-api-access-mx5z5\") pod \"mt-curl-7704-7369dc\" (UID: \"dde0d543-6b9c-4c39-b69d-1b9f341a518b\") " pod="opendatahub/mt-curl-7704-7369dc"
Sep 08 20:47:52.384610 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.384582    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-7369dc"
Sep 08 20:47:52.666692 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:52.666667    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7704-7369dc"]
Sep 08 20:47:52.667708 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:47:52.667681    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddde0d543_6b9c_4c39_b69d_1b9f341a518b.slice/crio-ced6161940ea1deba6bed0c1df7f53488b7de974be8a1b13214127cefed0520a WatchSource:0}: Error finding container ced6161940ea1deba6bed0c1df7f53488b7de974be8a1b13214127cefed0520a: Status 404 returned error can't find the container with id ced6161940ea1deba6bed0c1df7f53488b7de974be8a1b13214127cefed0520a
Sep 08 20:47:53.196467 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:53.196426    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7704-7369dc" event={"ID":"dde0d543-6b9c-4c39-b69d-1b9f341a518b","Type":"ContainerStarted","Data":"ced6161940ea1deba6bed0c1df7f53488b7de974be8a1b13214127cefed0520a"}
Sep 08 20:47:54.207020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:54.206973    2561 generic.go:356] "Generic (PLEG): container finished" podID="dde0d543-6b9c-4c39-b69d-1b9f341a518b" containerID="091126a6bfa810092e165fa26166414cff558e449949093a8764311eea152a22" exitCode=0
Sep 08 20:47:54.207453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:54.207044    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7704-7369dc" event={"ID":"dde0d543-6b9c-4c39-b69d-1b9f341a518b","Type":"ContainerDied","Data":"091126a6bfa810092e165fa26166414cff558e449949093a8764311eea152a22"}
Sep 08 20:47:55.358640 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:55.358608    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-7369dc"
Sep 08 20:47:55.390748 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:55.390717    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7704-7369dc_dde0d543-6b9c-4c39-b69d-1b9f341a518b/mt-curl-7704-7369dc/0.log"
Sep 08 20:47:55.415371 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:55.415329    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7704-7369dc"]
Sep 08 20:47:55.418978 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:55.418948    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7704-7369dc"]
Sep 08 20:47:55.479283 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:55.479196    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx5z5\" (UniqueName: \"kubernetes.io/projected/dde0d543-6b9c-4c39-b69d-1b9f341a518b-kube-api-access-mx5z5\") pod \"dde0d543-6b9c-4c39-b69d-1b9f341a518b\" (UID: \"dde0d543-6b9c-4c39-b69d-1b9f341a518b\") "
Sep 08 20:47:55.481390 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:55.481365    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde0d543-6b9c-4c39-b69d-1b9f341a518b-kube-api-access-mx5z5" (OuterVolumeSpecName: "kube-api-access-mx5z5") pod "dde0d543-6b9c-4c39-b69d-1b9f341a518b" (UID: "dde0d543-6b9c-4c39-b69d-1b9f341a518b"). InnerVolumeSpecName "kube-api-access-mx5z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:47:55.580332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:55.580296    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx5z5\" (UniqueName: \"kubernetes.io/projected/dde0d543-6b9c-4c39-b69d-1b9f341a518b-kube-api-access-mx5z5\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:47:56.225518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:56.225484    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-7369dc"
Sep 08 20:47:56.225518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:56.225516    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ced6161940ea1deba6bed0c1df7f53488b7de974be8a1b13214127cefed0520a"
Sep 08 20:47:56.236329 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:47:56.236287    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7704-7369dc\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="dde0d543-6b9c-4c39-b69d-1b9f341a518b" pod="opendatahub/mt-curl-7704-7369dc"
Sep 08 20:47:56.365733 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:47:56.365684    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde0d543-6b9c-4c39-b69d-1b9f341a518b" path="/var/lib/kubelet/pods/dde0d543-6b9c-4c39-b69d-1b9f341a518b/volumes"
Sep 08 20:48:14.702216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:14.702185    2561 scope.go:117] "RemoveContainer" containerID="9259907cb64b3d2aa62bf456efcb3f235ef8f2f45fd7a7b2a243a4922a46c668"
Sep 08 20:48:35.500983 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.500949    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"]
Sep 08 20:48:35.501520 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.501501    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde0d543-6b9c-4c39-b69d-1b9f341a518b" containerName="mt-curl-7704-7369dc"
Sep 08 20:48:35.501520 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.501519    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde0d543-6b9c-4c39-b69d-1b9f341a518b" containerName="mt-curl-7704-7369dc"
Sep 08 20:48:35.501660 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.501605    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde0d543-6b9c-4c39-b69d-1b9f341a518b" containerName="mt-curl-7704-7369dc"
Sep 08 20:48:35.507586 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.507553    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:35.510207 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.510169    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ce2cc8-dockercfg-f4jg2\""
Sep 08 20:48:35.510399 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.510172    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ce2cc8\""
Sep 08 20:48:35.515491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.515460    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"]
Sep 08 20:48:35.549590 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.549555    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"]
Sep 08 20:48:35.553770 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.553749    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:35.566388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.566360    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"]
Sep 08 20:48:35.679274 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.679243    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt9fj\" (UniqueName: \"kubernetes.io/projected/00921dae-10ad-4dba-91d1-6c73511cb19c-kube-api-access-bt9fj\") pod \"payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb\" (UID: \"00921dae-10ad-4dba-91d1-6c73511cb19c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:35.679540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.679280    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk\" (UID: \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:35.679540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.679394    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00921dae-10ad-4dba-91d1-6c73511cb19c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb\" (UID: \"00921dae-10ad-4dba-91d1-6c73511cb19c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:35.679540 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.679470    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mvj8\" (UniqueName: \"kubernetes.io/projected/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-kube-api-access-8mvj8\") pod \"payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk\" (UID: \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:35.781113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.781020    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt9fj\" (UniqueName: \"kubernetes.io/projected/00921dae-10ad-4dba-91d1-6c73511cb19c-kube-api-access-bt9fj\") pod \"payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb\" (UID: \"00921dae-10ad-4dba-91d1-6c73511cb19c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:35.781113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.781059    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk\" (UID: \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:35.781361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.781122    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00921dae-10ad-4dba-91d1-6c73511cb19c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb\" (UID: \"00921dae-10ad-4dba-91d1-6c73511cb19c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:35.781361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.781157    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mvj8\" (UniqueName: \"kubernetes.io/projected/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-kube-api-access-8mvj8\") pod \"payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk\" (UID: \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:35.781826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.781805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk\" (UID: \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:35.781887 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.781870    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00921dae-10ad-4dba-91d1-6c73511cb19c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb\" (UID: \"00921dae-10ad-4dba-91d1-6c73511cb19c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:35.789947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.789907    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt9fj\" (UniqueName: \"kubernetes.io/projected/00921dae-10ad-4dba-91d1-6c73511cb19c-kube-api-access-bt9fj\") pod \"payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb\" (UID: \"00921dae-10ad-4dba-91d1-6c73511cb19c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:35.790093 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.789959    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mvj8\" (UniqueName: \"kubernetes.io/projected/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-kube-api-access-8mvj8\") pod \"payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk\" (UID: \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:35.822752 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.822709    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:35.865704 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:35.865605    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:36.101581 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.101549    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"]
Sep 08 20:48:36.102114 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:48:36.102092    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d7f92e_d55c_4edf_b8ce_5d4fe1524f63.slice/crio-77ea1e71d3160c88cd72a3eab921308e81e9e136c5bacaea70c6374e4e968ea9 WatchSource:0}: Error finding container 77ea1e71d3160c88cd72a3eab921308e81e9e136c5bacaea70c6374e4e968ea9: Status 404 returned error can't find the container with id 77ea1e71d3160c88cd72a3eab921308e81e9e136c5bacaea70c6374e4e968ea9
Sep 08 20:48:36.165403 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:48:36.165348    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00921dae_10ad_4dba_91d1_6c73511cb19c.slice/crio-19e24827ef314511b5ff8a8fbb016c7e91bd121a21db92b80eda8e09967bf814 WatchSource:0}: Error finding container 19e24827ef314511b5ff8a8fbb016c7e91bd121a21db92b80eda8e09967bf814: Status 404 returned error can't find the container with id 19e24827ef314511b5ff8a8fbb016c7e91bd121a21db92b80eda8e09967bf814
Sep 08 20:48:36.165885 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.165835    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"]
Sep 08 20:48:36.514476 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.514433    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb" event={"ID":"00921dae-10ad-4dba-91d1-6c73511cb19c","Type":"ContainerStarted","Data":"7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b"}
Sep 08 20:48:36.514972 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.514501    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb" event={"ID":"00921dae-10ad-4dba-91d1-6c73511cb19c","Type":"ContainerStarted","Data":"19e24827ef314511b5ff8a8fbb016c7e91bd121a21db92b80eda8e09967bf814"}
Sep 08 20:48:36.514972 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.514547    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:36.516414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.516383    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk" event={"ID":"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63","Type":"ContainerStarted","Data":"12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848"}
Sep 08 20:48:36.516562 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.516436    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk" event={"ID":"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63","Type":"ContainerStarted","Data":"77ea1e71d3160c88cd72a3eab921308e81e9e136c5bacaea70c6374e4e968ea9"}
Sep 08 20:48:36.516606 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.516589    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:36.534129 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.534056    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb" podStartSLOduration=1.5340336350000001 podStartE2EDuration="1.534033635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:48:36.531473284 +0000 UTC m=+1402.696643865" watchObservedRunningTime="2026-09-08 20:48:36.534033635 +0000 UTC m=+1402.699204212"
Sep 08 20:48:36.550315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:36.550247    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk" podStartSLOduration=1.5502310430000001 podStartE2EDuration="1.550231043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:48:36.549831347 +0000 UTC m=+1402.715001924" watchObservedRunningTime="2026-09-08 20:48:36.550231043 +0000 UTC m=+1402.715401621"
Sep 08 20:48:47.526830 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:47.526795    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:48:47.527453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:47.526953    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:48:59.423124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.423088    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"]
Sep 08 20:48:59.427075 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.427052    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.429702 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.429682    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-792dc6-openshift-default-dockercfg-b6656\""
Sep 08 20:48:59.442171 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.442135    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"]
Sep 08 20:48:59.526485 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526444    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-certs\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.526664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526489    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-socket\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.526664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-envoy\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.526664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526567    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-data\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.526664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526610    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-podinfo\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.526664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526640    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25eadd26-de1a-4d62-8ba7-e4e306806b80-istiod-ca-cert\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.526855 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526732    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-credential-socket\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.526855 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526793    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-token\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.526855 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.526821    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsnkw\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-kube-api-access-bsnkw\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.627781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.627740    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-envoy\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.627781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.627781    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-data\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.627816    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-podinfo\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.627839    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25eadd26-de1a-4d62-8ba7-e4e306806b80-istiod-ca-cert\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.627871    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-credential-socket\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.627940    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-token\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.627960    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsnkw\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-kube-api-access-bsnkw\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.628030    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-certs\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.628060    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-socket\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.628287    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-data\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.628369    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-credential-socket\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.628411    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-socket\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.628614 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.628587    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-certs\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.629286 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.629262    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25eadd26-de1a-4d62-8ba7-e4e306806b80-istiod-ca-cert\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.630306 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.630284    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-envoy\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.630465 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.630444    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-podinfo\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.635887 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.635850    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsnkw\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-kube-api-access-bsnkw\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.636117 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.636099    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-token\") pod \"e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") " pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:48:59.745601 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:48:59.745503    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:00.030509 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:00.030470    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"]
Sep 08 20:49:00.031596 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:49:00.031478    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25eadd26_de1a_4d62_8ba7_e4e306806b80.slice/crio-cffb79d9a7cc23070eb5f796328bcbbca1e085b222e38d90afe99ef28516cb9c WatchSource:0}: Error finding container cffb79d9a7cc23070eb5f796328bcbbca1e085b222e38d90afe99ef28516cb9c: Status 404 returned error can't find the container with id cffb79d9a7cc23070eb5f796328bcbbca1e085b222e38d90afe99ef28516cb9c
Sep 08 20:49:00.034739 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:00.034702    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:49:00.034851 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:00.034774    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:49:00.034851 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:00.034807    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:49:00.694807 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:00.694769    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" event={"ID":"25eadd26-de1a-4d62-8ba7-e4e306806b80","Type":"ContainerStarted","Data":"f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4"}
Sep 08 20:49:00.695246 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:00.694822    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" event={"ID":"25eadd26-de1a-4d62-8ba7-e4e306806b80","Type":"ContainerStarted","Data":"cffb79d9a7cc23070eb5f796328bcbbca1e085b222e38d90afe99ef28516cb9c"}
Sep 08 20:49:00.722062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:00.721992    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" podStartSLOduration=1.7219695320000001 podStartE2EDuration="1.721969532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:49:00.71780935 +0000 UTC m=+1426.882979927" watchObservedRunningTime="2026-09-08 20:49:00.721969532 +0000 UTC m=+1426.887140109"
Sep 08 20:49:00.746211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:00.746171    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:01.752951 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:01.752916    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:02.709763 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:02.709732    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:02.710794 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:02.710776    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:14.732407 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:14.732312    2561 scope.go:117] "RemoveContainer" containerID="449f951459e8aff30af7be3b08a720adc6fc543b739a216e233ffd9764c30b75"
Sep 08 20:49:14.748042 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:14.748011    2561 scope.go:117] "RemoveContainer" containerID="384fce4a5e3e8111cc1a3ec1b679758f291fcba94f39d27c5fbd6be01ec9c638"
Sep 08 20:49:22.872725 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:22.872645    2561 generic.go:356] "Generic (PLEG): container finished" podID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerID="f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4" exitCode=0
Sep 08 20:49:22.872725 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:22.872693    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" event={"ID":"25eadd26-de1a-4d62-8ba7-e4e306806b80","Type":"ContainerDied","Data":"f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4"}
Sep 08 20:49:22.873341 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:22.873304    2561 scope.go:117] "RemoveContainer" containerID="f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4"
Sep 08 20:49:22.874715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:22.874681    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:49:22.874852 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:22.874773    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:49:22.874852 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:22.874822    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:49:23.882126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:23.882087    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" event={"ID":"25eadd26-de1a-4d62-8ba7-e4e306806b80","Type":"ContainerStarted","Data":"2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4"}
Sep 08 20:49:24.713445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.713409    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7704-38ff7a"]
Sep 08 20:49:24.717640 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.717621    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-38ff7a"
Sep 08 20:49:24.720558 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.720535    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qqm77\""
Sep 08 20:49:24.724996 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.724815    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7704-38ff7a"]
Sep 08 20:49:24.747916 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.746537    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:24.747916 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.746674    2561 patch_prober.go:28] interesting pod/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Sep 08 20:49:24.747916 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.746719    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Sep 08 20:49:24.779805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.779768    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v77b\" (UniqueName: \"kubernetes.io/projected/2ea04751-3031-43cd-85f2-d2e3f83f0d51-kube-api-access-5v77b\") pod \"mt-curl-7704-38ff7a\" (UID: \"2ea04751-3031-43cd-85f2-d2e3f83f0d51\") " pod="opendatahub/mt-curl-7704-38ff7a"
Sep 08 20:49:24.880864 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.880826    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v77b\" (UniqueName: \"kubernetes.io/projected/2ea04751-3031-43cd-85f2-d2e3f83f0d51-kube-api-access-5v77b\") pod \"mt-curl-7704-38ff7a\" (UID: \"2ea04751-3031-43cd-85f2-d2e3f83f0d51\") " pod="opendatahub/mt-curl-7704-38ff7a"
Sep 08 20:49:24.893668 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:24.893632    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v77b\" (UniqueName: \"kubernetes.io/projected/2ea04751-3031-43cd-85f2-d2e3f83f0d51-kube-api-access-5v77b\") pod \"mt-curl-7704-38ff7a\" (UID: \"2ea04751-3031-43cd-85f2-d2e3f83f0d51\") " pod="opendatahub/mt-curl-7704-38ff7a"
Sep 08 20:49:25.030057 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:25.029965    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-38ff7a"
Sep 08 20:49:25.325866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:25.325839    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7704-38ff7a"]
Sep 08 20:49:25.326763 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:49:25.326733    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ea04751_3031_43cd_85f2_d2e3f83f0d51.slice/crio-03d8231fd3b83278b370a8e6a684d26e6b96149d705dbfb888de3e32aad0d6f4 WatchSource:0}: Error finding container 03d8231fd3b83278b370a8e6a684d26e6b96149d705dbfb888de3e32aad0d6f4: Status 404 returned error can't find the container with id 03d8231fd3b83278b370a8e6a684d26e6b96149d705dbfb888de3e32aad0d6f4
Sep 08 20:49:25.749920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:25.747322    2561 patch_prober.go:28] interesting pod/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Sep 08 20:49:25.749920 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:25.747391    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Sep 08 20:49:25.900736 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:25.900641    2561 generic.go:356] "Generic (PLEG): container finished" podID="2ea04751-3031-43cd-85f2-d2e3f83f0d51" containerID="d3e1713cddeaa6002599efb97b4cc48d80298601faf743d75b36c40eb80cc8f2" exitCode=0
Sep 08 20:49:25.901140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:25.900741    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7704-38ff7a" event={"ID":"2ea04751-3031-43cd-85f2-d2e3f83f0d51","Type":"ContainerDied","Data":"d3e1713cddeaa6002599efb97b4cc48d80298601faf743d75b36c40eb80cc8f2"}
Sep 08 20:49:25.901140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:25.900786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7704-38ff7a" event={"ID":"2ea04751-3031-43cd-85f2-d2e3f83f0d51","Type":"ContainerStarted","Data":"03d8231fd3b83278b370a8e6a684d26e6b96149d705dbfb888de3e32aad0d6f4"}
Sep 08 20:49:26.746521 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:26.746473    2561 patch_prober.go:28] interesting pod/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Sep 08 20:49:26.746745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:26.746553    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Sep 08 20:49:27.096191 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.096162    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-38ff7a"
Sep 08 20:49:27.150516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.150398    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7704-38ff7a_2ea04751-3031-43cd-85f2-d2e3f83f0d51/mt-curl-7704-38ff7a/0.log"
Sep 08 20:49:27.181954 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.181886    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7704-38ff7a"]
Sep 08 20:49:27.189667 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.189618    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7704-38ff7a"]
Sep 08 20:49:27.204183 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.204150    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v77b\" (UniqueName: \"kubernetes.io/projected/2ea04751-3031-43cd-85f2-d2e3f83f0d51-kube-api-access-5v77b\") pod \"2ea04751-3031-43cd-85f2-d2e3f83f0d51\" (UID: \"2ea04751-3031-43cd-85f2-d2e3f83f0d51\") "
Sep 08 20:49:27.206442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.206410    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ea04751-3031-43cd-85f2-d2e3f83f0d51-kube-api-access-5v77b" (OuterVolumeSpecName: "kube-api-access-5v77b") pod "2ea04751-3031-43cd-85f2-d2e3f83f0d51" (UID: "2ea04751-3031-43cd-85f2-d2e3f83f0d51"). InnerVolumeSpecName "kube-api-access-5v77b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:49:27.305449 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.305411    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v77b\" (UniqueName: \"kubernetes.io/projected/2ea04751-3031-43cd-85f2-d2e3f83f0d51-kube-api-access-5v77b\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:49:27.316848 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.316811    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7704-b6397c"]
Sep 08 20:49:27.317400 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.317377    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ea04751-3031-43cd-85f2-d2e3f83f0d51" containerName="mt-curl-7704-38ff7a"
Sep 08 20:49:27.317457 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.317403    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ea04751-3031-43cd-85f2-d2e3f83f0d51" containerName="mt-curl-7704-38ff7a"
Sep 08 20:49:27.317500 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.317491    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ea04751-3031-43cd-85f2-d2e3f83f0d51" containerName="mt-curl-7704-38ff7a"
Sep 08 20:49:27.320875 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.320795    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-b6397c"
Sep 08 20:49:27.328806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.328751    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7704-b6397c"]
Sep 08 20:49:27.406551 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.406507    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6stdl\" (UniqueName: \"kubernetes.io/projected/3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3-kube-api-access-6stdl\") pod \"mt-curl-7704-b6397c\" (UID: \"3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3\") " pod="opendatahub/mt-curl-7704-b6397c"
Sep 08 20:49:27.507837 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.507799    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6stdl\" (UniqueName: \"kubernetes.io/projected/3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3-kube-api-access-6stdl\") pod \"mt-curl-7704-b6397c\" (UID: \"3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3\") " pod="opendatahub/mt-curl-7704-b6397c"
Sep 08 20:49:27.517355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.517323    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6stdl\" (UniqueName: \"kubernetes.io/projected/3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3-kube-api-access-6stdl\") pod \"mt-curl-7704-b6397c\" (UID: \"3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3\") " pod="opendatahub/mt-curl-7704-b6397c"
Sep 08 20:49:27.651605 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.651564    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-b6397c"
Sep 08 20:49:27.749113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.748389    2561 patch_prober.go:28] interesting pod/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Sep 08 20:49:27.749113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.748470    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Sep 08 20:49:27.916798 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.916719    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-38ff7a"
Sep 08 20:49:27.916798 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.916737    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03d8231fd3b83278b370a8e6a684d26e6b96149d705dbfb888de3e32aad0d6f4"
Sep 08 20:49:27.929667 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:49:27.929615    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7704-38ff7a\" is forbidden: User \"system:node:ip-10-0-132-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-228.ec2.internal' and this object" podUID="2ea04751-3031-43cd-85f2-d2e3f83f0d51" pod="opendatahub/mt-curl-7704-38ff7a"
Sep 08 20:49:27.944120 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:27.943964    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7704-b6397c"]
Sep 08 20:49:28.368230 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:28.368191    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ea04751-3031-43cd-85f2-d2e3f83f0d51" path="/var/lib/kubelet/pods/2ea04751-3031-43cd-85f2-d2e3f83f0d51/volumes"
Sep 08 20:49:28.746268 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:28.746224    2561 patch_prober.go:28] interesting pod/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Sep 08 20:49:28.746489 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:28.746293    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Sep 08 20:49:28.927073 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:28.927031    2561 generic.go:356] "Generic (PLEG): container finished" podID="3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3" containerID="b7a17221c56322846436c5345158c0d476c9a27b3a0afefaea4261ed262fbc84" exitCode=0
Sep 08 20:49:28.927254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:28.927106    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7704-b6397c" event={"ID":"3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3","Type":"ContainerDied","Data":"b7a17221c56322846436c5345158c0d476c9a27b3a0afefaea4261ed262fbc84"}
Sep 08 20:49:28.927254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:28.927149    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7704-b6397c" event={"ID":"3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3","Type":"ContainerStarted","Data":"34d06002c891fa792fb70bd652741470fdbcc87169c0f7de42f30e4333ee5392"}
Sep 08 20:49:29.751028 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:29.750988    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:29.751475 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:29.751378    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:29.752262 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:29.752243    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:49:30.084023 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.083997    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-b6397c"
Sep 08 20:49:30.103139 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.103112    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7704-b6397c_3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3/mt-curl-7704-b6397c/0.log"
Sep 08 20:49:30.126358 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.126321    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7704-b6397c"]
Sep 08 20:49:30.131925 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.131859    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7704-b6397c"]
Sep 08 20:49:30.137152 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.137115    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6stdl\" (UniqueName: \"kubernetes.io/projected/3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3-kube-api-access-6stdl\") pod \"3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3\" (UID: \"3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3\") "
Sep 08 20:49:30.139416 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.139389    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3-kube-api-access-6stdl" (OuterVolumeSpecName: "kube-api-access-6stdl") pod "3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3" (UID: "3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3"). InnerVolumeSpecName "kube-api-access-6stdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:49:30.238498 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.238452    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6stdl\" (UniqueName: \"kubernetes.io/projected/3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3-kube-api-access-6stdl\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:49:30.365221 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.365131    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3" path="/var/lib/kubelet/pods/3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3/volumes"
Sep 08 20:49:30.783785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.783746    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"]
Sep 08 20:49:30.784522 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.784499    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3" containerName="mt-curl-7704-b6397c"
Sep 08 20:49:30.784624 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.784526    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3" containerName="mt-curl-7704-b6397c"
Sep 08 20:49:30.784681 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.784660    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3ae7ec-c31d-4ea6-b76f-c1253d064bb3" containerName="mt-curl-7704-b6397c"
Sep 08 20:49:30.799003 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.798943    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"
Sep 08 20:49:30.800694 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.800667    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"]
Sep 08 20:49:30.801681 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.801665    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:49:30.845361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.845316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f16b0c-a817-4271-b3f2-da6d4c1095e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb\" (UID: \"10f16b0c-a817-4271-b3f2-da6d4c1095e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"
Sep 08 20:49:30.943957 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.943929    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7704-b6397c"
Sep 08 20:49:30.943957 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.943916    2561 scope.go:117] "RemoveContainer" containerID="b7a17221c56322846436c5345158c0d476c9a27b3a0afefaea4261ed262fbc84"
Sep 08 20:49:30.945877 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.945854    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f16b0c-a817-4271-b3f2-da6d4c1095e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb\" (UID: \"10f16b0c-a817-4271-b3f2-da6d4c1095e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"
Sep 08 20:49:30.946579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:30.946556    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f16b0c-a817-4271-b3f2-da6d4c1095e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb\" (UID: \"10f16b0c-a817-4271-b3f2-da6d4c1095e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"
Sep 08 20:49:31.111344 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:31.111254    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"
Sep 08 20:49:31.386412 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:31.386384    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"]
Sep 08 20:49:31.387384 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:49:31.387360    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f16b0c_a817_4271_b3f2_da6d4c1095e0.slice/crio-88b529f448388f28b2d0404979b493432fe81222f3845f15813411741a7e6c0a WatchSource:0}: Error finding container 88b529f448388f28b2d0404979b493432fe81222f3845f15813411741a7e6c0a: Status 404 returned error can't find the container with id 88b529f448388f28b2d0404979b493432fe81222f3845f15813411741a7e6c0a
Sep 08 20:49:31.954070 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:31.954033    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb" event={"ID":"10f16b0c-a817-4271-b3f2-da6d4c1095e0","Type":"ContainerStarted","Data":"331dade7430abdce3944cfd9488eaebdc78954d795ddb320fa0f033a6a5714b0"}
Sep 08 20:49:31.954560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:31.954094    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb" event={"ID":"10f16b0c-a817-4271-b3f2-da6d4c1095e0","Type":"ContainerStarted","Data":"88b529f448388f28b2d0404979b493432fe81222f3845f15813411741a7e6c0a"}
Sep 08 20:49:31.969552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:31.969498    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb" podStartSLOduration=1.96948297 podStartE2EDuration="1.96948297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:49:31.968182052 +0000 UTC m=+1458.133352629" watchObservedRunningTime="2026-09-08 20:49:31.96948297 +0000 UTC m=+1458.134653549"
Sep 08 20:49:32.962142 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:32.962106    2561 generic.go:356] "Generic (PLEG): container finished" podID="10f16b0c-a817-4271-b3f2-da6d4c1095e0" containerID="331dade7430abdce3944cfd9488eaebdc78954d795ddb320fa0f033a6a5714b0" exitCode=0
Sep 08 20:49:32.962548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:32.962191    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb" event={"ID":"10f16b0c-a817-4271-b3f2-da6d4c1095e0","Type":"ContainerDied","Data":"331dade7430abdce3944cfd9488eaebdc78954d795ddb320fa0f033a6a5714b0"}
Sep 08 20:49:34.128343 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:34.128314    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"
Sep 08 20:49:34.177745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:34.177705    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f16b0c-a817-4271-b3f2-da6d4c1095e0-maas-api-service-ca\") pod \"10f16b0c-a817-4271-b3f2-da6d4c1095e0\" (UID: \"10f16b0c-a817-4271-b3f2-da6d4c1095e0\") "
Sep 08 20:49:34.178155 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:34.178134    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10f16b0c-a817-4271-b3f2-da6d4c1095e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "10f16b0c-a817-4271-b3f2-da6d4c1095e0" (UID: "10f16b0c-a817-4271-b3f2-da6d4c1095e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:49:34.279425 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:34.279344    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f16b0c-a817-4271-b3f2-da6d4c1095e0-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:49:34.986295 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:34.986261    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"
Sep 08 20:49:34.986508 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:34.986295    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb" event={"ID":"10f16b0c-a817-4271-b3f2-da6d4c1095e0","Type":"ContainerDied","Data":"88b529f448388f28b2d0404979b493432fe81222f3845f15813411741a7e6c0a"}
Sep 08 20:49:34.986508 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:34.986381    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88b529f448388f28b2d0404979b493432fe81222f3845f15813411741a7e6c0a"
Sep 08 20:49:46.743280 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:46.743235    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"]
Sep 08 20:49:46.743973 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:46.743488    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb" podUID="00921dae-10ad-4dba-91d1-6c73511cb19c" containerName="payload-processing" containerID="cri-o://7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b" gracePeriod=30
Sep 08 20:49:46.747473 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:46.747432    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"]
Sep 08 20:49:46.748037 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:46.747985    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk" podUID="22d7f92e-d55c-4edf-b8ce-5d4fe1524f63" containerName="payload-pre-processing" containerID="cri-o://12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848" gracePeriod=30
Sep 08 20:49:46.911736 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:46.911695    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"]
Sep 08 20:49:46.918461 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:46.918419    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce2cc8-skldb"]
Sep 08 20:49:47.023608 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.023578    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:49:47.069465 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.069428    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:49:47.080748 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.080590    2561 generic.go:356] "Generic (PLEG): container finished" podID="22d7f92e-d55c-4edf-b8ce-5d4fe1524f63" containerID="12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848" exitCode=0
Sep 08 20:49:47.081009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.080776    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk" event={"ID":"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63","Type":"ContainerDied","Data":"12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848"}
Sep 08 20:49:47.081009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.080917    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk" event={"ID":"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63","Type":"ContainerDied","Data":"77ea1e71d3160c88cd72a3eab921308e81e9e136c5bacaea70c6374e4e968ea9"}
Sep 08 20:49:47.081009 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.080981    2561 scope.go:117] "RemoveContainer" containerID="12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848"
Sep 08 20:49:47.084361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.081870    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"
Sep 08 20:49:47.084361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.084245    2561 generic.go:356] "Generic (PLEG): container finished" podID="00921dae-10ad-4dba-91d1-6c73511cb19c" containerID="7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b" exitCode=0
Sep 08 20:49:47.084361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.084366    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb" event={"ID":"00921dae-10ad-4dba-91d1-6c73511cb19c","Type":"ContainerDied","Data":"7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b"}
Sep 08 20:49:47.084361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.084400    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"
Sep 08 20:49:47.084715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.084417    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb" event={"ID":"00921dae-10ad-4dba-91d1-6c73511cb19c","Type":"ContainerDied","Data":"19e24827ef314511b5ff8a8fbb016c7e91bd121a21db92b80eda8e09967bf814"}
Sep 08 20:49:47.097168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.097141    2561 scope.go:117] "RemoveContainer" containerID="12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848"
Sep 08 20:49:47.097588 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:49:47.097556    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848\": container with ID starting with 12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848 not found: ID does not exist" containerID="12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848"
Sep 08 20:49:47.097713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.097597    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848"} err="failed to get container status \"12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848\": rpc error: code = NotFound desc = could not find container \"12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848\": container with ID starting with 12673ea2064a6eb718aab600967415f115199d9e2a409836df2d5c27f7deb848 not found: ID does not exist"
Sep 08 20:49:47.097713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.097617    2561 scope.go:117] "RemoveContainer" containerID="7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b"
Sep 08 20:49:47.110184 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.110158    2561 scope.go:117] "RemoveContainer" containerID="7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b"
Sep 08 20:49:47.110539 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:49:47.110516    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b\": container with ID starting with 7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b not found: ID does not exist" containerID="7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b"
Sep 08 20:49:47.110665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.110544    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b"} err="failed to get container status \"7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b\": rpc error: code = NotFound desc = could not find container \"7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b\": container with ID starting with 7a9af8610b3d7282b2318e0264cb564d5a06341b63fd0986811cd3b68054201b not found: ID does not exist"
Sep 08 20:49:47.111871 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.111842    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00921dae-10ad-4dba-91d1-6c73511cb19c-plugins-config-volume\") pod \"00921dae-10ad-4dba-91d1-6c73511cb19c\" (UID: \"00921dae-10ad-4dba-91d1-6c73511cb19c\") "
Sep 08 20:49:47.112020 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.112008    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt9fj\" (UniqueName: \"kubernetes.io/projected/00921dae-10ad-4dba-91d1-6c73511cb19c-kube-api-access-bt9fj\") pod \"00921dae-10ad-4dba-91d1-6c73511cb19c\" (UID: \"00921dae-10ad-4dba-91d1-6c73511cb19c\") "
Sep 08 20:49:47.112280 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.112255    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00921dae-10ad-4dba-91d1-6c73511cb19c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00921dae-10ad-4dba-91d1-6c73511cb19c" (UID: "00921dae-10ad-4dba-91d1-6c73511cb19c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:49:47.112458 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.112437    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00921dae-10ad-4dba-91d1-6c73511cb19c-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.114178 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.114155    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00921dae-10ad-4dba-91d1-6c73511cb19c-kube-api-access-bt9fj" (OuterVolumeSpecName: "kube-api-access-bt9fj") pod "00921dae-10ad-4dba-91d1-6c73511cb19c" (UID: "00921dae-10ad-4dba-91d1-6c73511cb19c"). InnerVolumeSpecName "kube-api-access-bt9fj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:49:47.213071 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.213038    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mvj8\" (UniqueName: \"kubernetes.io/projected/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-kube-api-access-8mvj8\") pod \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\" (UID: \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\") "
Sep 08 20:49:47.213269 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.213145    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-plugins-config-volume\") pod \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\" (UID: \"22d7f92e-d55c-4edf-b8ce-5d4fe1524f63\") "
Sep 08 20:49:47.213479 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.213462    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt9fj\" (UniqueName: \"kubernetes.io/projected/00921dae-10ad-4dba-91d1-6c73511cb19c-kube-api-access-bt9fj\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.213558 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.213532    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "22d7f92e-d55c-4edf-b8ce-5d4fe1524f63" (UID: "22d7f92e-d55c-4edf-b8ce-5d4fe1524f63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:49:47.215393 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.215365    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-kube-api-access-8mvj8" (OuterVolumeSpecName: "kube-api-access-8mvj8") pod "22d7f92e-d55c-4edf-b8ce-5d4fe1524f63" (UID: "22d7f92e-d55c-4edf-b8ce-5d4fe1524f63"). InnerVolumeSpecName "kube-api-access-8mvj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:49:47.314313 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.314217    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.314313 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.314254    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mvj8\" (UniqueName: \"kubernetes.io/projected/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63-kube-api-access-8mvj8\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.405510 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.405470    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"]
Sep 08 20:49:47.410828 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.410789    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce2cc8-5d686cd974-7v7wk"]
Sep 08 20:49:47.422790 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.422748    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"]
Sep 08 20:49:47.428157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:47.428123    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce2cc8-6998c744f5-x2bqb"]
Sep 08 20:49:48.364850 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:48.364816    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00921dae-10ad-4dba-91d1-6c73511cb19c" path="/var/lib/kubelet/pods/00921dae-10ad-4dba-91d1-6c73511cb19c/volumes"
Sep 08 20:49:48.365258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:48.365229    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10f16b0c-a817-4271-b3f2-da6d4c1095e0" path="/var/lib/kubelet/pods/10f16b0c-a817-4271-b3f2-da6d4c1095e0/volumes"
Sep 08 20:49:48.365561 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:48.365546    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22d7f92e-d55c-4edf-b8ce-5d4fe1524f63" path="/var/lib/kubelet/pods/22d7f92e-d55c-4edf-b8ce-5d4fe1524f63/volumes"
Sep 08 20:49:53.824861 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.824820    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"]
Sep 08 20:49:53.825422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825407    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22d7f92e-d55c-4edf-b8ce-5d4fe1524f63" containerName="payload-pre-processing"
Sep 08 20:49:53.825422 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825423    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="22d7f92e-d55c-4edf-b8ce-5d4fe1524f63" containerName="payload-pre-processing"
Sep 08 20:49:53.825511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825433    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00921dae-10ad-4dba-91d1-6c73511cb19c" containerName="payload-processing"
Sep 08 20:49:53.825511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825439    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="00921dae-10ad-4dba-91d1-6c73511cb19c" containerName="payload-processing"
Sep 08 20:49:53.825511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825459    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10f16b0c-a817-4271-b3f2-da6d4c1095e0" containerName="revoke-keys"
Sep 08 20:49:53.825511 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825465    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="10f16b0c-a817-4271-b3f2-da6d4c1095e0" containerName="revoke-keys"
Sep 08 20:49:53.825646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825545    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="22d7f92e-d55c-4edf-b8ce-5d4fe1524f63" containerName="payload-pre-processing"
Sep 08 20:49:53.825646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825552    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="10f16b0c-a817-4271-b3f2-da6d4c1095e0" containerName="revoke-keys"
Sep 08 20:49:53.825646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.825562    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="00921dae-10ad-4dba-91d1-6c73511cb19c" containerName="payload-processing"
Sep 08 20:49:53.830415 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.830399    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"
Sep 08 20:49:53.833250 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.833035    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:49:53.840562 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.840537    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"]
Sep 08 20:49:53.982242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:53.982196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f010056-242c-41c8-9397-f449c942a510-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh\" (UID: \"6f010056-242c-41c8-9397-f449c942a510\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"
Sep 08 20:49:54.083280 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:54.083180    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f010056-242c-41c8-9397-f449c942a510-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh\" (UID: \"6f010056-242c-41c8-9397-f449c942a510\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"
Sep 08 20:49:54.083973 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:54.083952    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f010056-242c-41c8-9397-f449c942a510-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh\" (UID: \"6f010056-242c-41c8-9397-f449c942a510\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"
Sep 08 20:49:54.147453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:54.147414    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"
Sep 08 20:49:54.434077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:54.434045    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"]
Sep 08 20:49:54.434562 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:49:54.434522    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f010056_242c_41c8_9397_f449c942a510.slice/crio-0b2e84cf2f62d2da97fa7047e35bfdb0d666c5429cb08aa4847a549abecd5a54 WatchSource:0}: Error finding container 0b2e84cf2f62d2da97fa7047e35bfdb0d666c5429cb08aa4847a549abecd5a54: Status 404 returned error can't find the container with id 0b2e84cf2f62d2da97fa7047e35bfdb0d666c5429cb08aa4847a549abecd5a54
Sep 08 20:49:55.151531 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:55.151498    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh" event={"ID":"6f010056-242c-41c8-9397-f449c942a510","Type":"ContainerStarted","Data":"2828974e2d7b8c96b2a367d637a0f8f4bf12bc4b81ff99e67d40ad17678714f9"}
Sep 08 20:49:55.151984 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:55.151551    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh" event={"ID":"6f010056-242c-41c8-9397-f449c942a510","Type":"ContainerStarted","Data":"0b2e84cf2f62d2da97fa7047e35bfdb0d666c5429cb08aa4847a549abecd5a54"}
Sep 08 20:49:55.169122 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:55.169073    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh" podStartSLOduration=2.169054025 podStartE2EDuration="2.169054025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:49:55.167479744 +0000 UTC m=+1481.332650327" watchObservedRunningTime="2026-09-08 20:49:55.169054025 +0000 UTC m=+1481.334224601"
Sep 08 20:49:56.160355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:56.160317    2561 generic.go:356] "Generic (PLEG): container finished" podID="6f010056-242c-41c8-9397-f449c942a510" containerID="2828974e2d7b8c96b2a367d637a0f8f4bf12bc4b81ff99e67d40ad17678714f9" exitCode=0
Sep 08 20:49:56.160748 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:56.160402    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh" event={"ID":"6f010056-242c-41c8-9397-f449c942a510","Type":"ContainerDied","Data":"2828974e2d7b8c96b2a367d637a0f8f4bf12bc4b81ff99e67d40ad17678714f9"}
Sep 08 20:49:57.305462 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:57.305432    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"
Sep 08 20:49:57.419388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:57.419298    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f010056-242c-41c8-9397-f449c942a510-maas-api-service-ca\") pod \"6f010056-242c-41c8-9397-f449c942a510\" (UID: \"6f010056-242c-41c8-9397-f449c942a510\") "
Sep 08 20:49:57.419658 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:57.419636    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f010056-242c-41c8-9397-f449c942a510-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f010056-242c-41c8-9397-f449c942a510" (UID: "6f010056-242c-41c8-9397-f449c942a510"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:49:57.521227 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:57.521192    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f010056-242c-41c8-9397-f449c942a510-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.175612 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:58.175572    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh" event={"ID":"6f010056-242c-41c8-9397-f449c942a510","Type":"ContainerDied","Data":"0b2e84cf2f62d2da97fa7047e35bfdb0d666c5429cb08aa4847a549abecd5a54"}
Sep 08 20:49:58.175612 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:58.175590    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"
Sep 08 20:49:58.175837 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:49:58.175651    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b2e84cf2f62d2da97fa7047e35bfdb0d666c5429cb08aa4847a549abecd5a54"
Sep 08 20:50:09.515570 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:09.515532    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"]
Sep 08 20:50:09.518581 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:09.518541    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-792dc6-fxdsh"]
Sep 08 20:50:10.365107 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:10.365065    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f010056-242c-41c8-9397-f449c942a510" path="/var/lib/kubelet/pods/6f010056-242c-41c8-9397-f449c942a510/volumes"
Sep 08 20:50:14.504154 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:14.504120    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:50:14.507842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:14.507819    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:50:14.784526 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:14.784429    2561 scope.go:117] "RemoveContainer" containerID="85ce4408c247b25ef669003759c1031c6ef61745d06d5f9e55ea434cc5b3550b"
Sep 08 20:50:16.003015 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.002976    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"]
Sep 08 20:50:16.003443 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.003314    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy" containerID="cri-o://2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4" gracePeriod=30
Sep 08 20:50:16.404715 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.404685    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"]
Sep 08 20:50:16.405231 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.405215    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f010056-242c-41c8-9397-f449c942a510" containerName="revoke-keys"
Sep 08 20:50:16.405292 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.405234    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f010056-242c-41c8-9397-f449c942a510" containerName="revoke-keys"
Sep 08 20:50:16.405329 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.405322    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f010056-242c-41c8-9397-f449c942a510" containerName="revoke-keys"
Sep 08 20:50:16.410123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.410107    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"
Sep 08 20:50:16.413783 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.413758    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:50:16.428209 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.428180    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"]
Sep 08 20:50:16.508282 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.508243    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fbdadce-ecad-47c5-a403-a77515109228-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7\" (UID: \"6fbdadce-ecad-47c5-a403-a77515109228\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"
Sep 08 20:50:16.608998 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.608962    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fbdadce-ecad-47c5-a403-a77515109228-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7\" (UID: \"6fbdadce-ecad-47c5-a403-a77515109228\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"
Sep 08 20:50:16.609601 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.609582    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fbdadce-ecad-47c5-a403-a77515109228-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7\" (UID: \"6fbdadce-ecad-47c5-a403-a77515109228\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"
Sep 08 20:50:16.721443 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:16.721343    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"
Sep 08 20:50:17.000532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:17.000498    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"]
Sep 08 20:50:17.001976 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:50:17.001949    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fbdadce_ecad_47c5_a403_a77515109228.slice/crio-4922d175409bd246ae8338f608b7fc150dc7adf3531f8e0cacae2ec26460c3db WatchSource:0}: Error finding container 4922d175409bd246ae8338f608b7fc150dc7adf3531f8e0cacae2ec26460c3db: Status 404 returned error can't find the container with id 4922d175409bd246ae8338f608b7fc150dc7adf3531f8e0cacae2ec26460c3db
Sep 08 20:50:17.308341 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:17.308259    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7" event={"ID":"6fbdadce-ecad-47c5-a403-a77515109228","Type":"ContainerStarted","Data":"4922d175409bd246ae8338f608b7fc150dc7adf3531f8e0cacae2ec26460c3db"}
Sep 08 20:50:18.316789 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:18.316751    2561 generic.go:356] "Generic (PLEG): container finished" podID="6fbdadce-ecad-47c5-a403-a77515109228" containerID="b04a69bbce7ed645656e00563121613fea3f7dab58f4e16079546bc22f9ceded" exitCode=0
Sep 08 20:50:18.317244 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:18.316831    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7" event={"ID":"6fbdadce-ecad-47c5-a403-a77515109228","Type":"ContainerDied","Data":"b04a69bbce7ed645656e00563121613fea3f7dab58f4e16079546bc22f9ceded"}
Sep 08 20:50:19.466745 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:19.466713    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"
Sep 08 20:50:19.539833 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:19.539794    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fbdadce-ecad-47c5-a403-a77515109228-maas-api-service-ca\") pod \"6fbdadce-ecad-47c5-a403-a77515109228\" (UID: \"6fbdadce-ecad-47c5-a403-a77515109228\") "
Sep 08 20:50:19.540203 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:19.540178    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fbdadce-ecad-47c5-a403-a77515109228-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6fbdadce-ecad-47c5-a403-a77515109228" (UID: "6fbdadce-ecad-47c5-a403-a77515109228"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:19.640810 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:19.640724    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fbdadce-ecad-47c5-a403-a77515109228-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:20.333077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:20.332831    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7" event={"ID":"6fbdadce-ecad-47c5-a403-a77515109228","Type":"ContainerDied","Data":"4922d175409bd246ae8338f608b7fc150dc7adf3531f8e0cacae2ec26460c3db"}
Sep 08 20:50:20.333077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:20.333074    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"
Sep 08 20:50:20.333341 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:20.333117    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4922d175409bd246ae8338f608b7fc150dc7adf3531f8e0cacae2ec26460c3db"
Sep 08 20:50:21.283048 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.283021    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:50:21.345470 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.345426    2561 generic.go:356] "Generic (PLEG): container finished" podID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerID="2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4" exitCode=0
Sep 08 20:50:21.345636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.345480    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"
Sep 08 20:50:21.345636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.345514    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" event={"ID":"25eadd26-de1a-4d62-8ba7-e4e306806b80","Type":"ContainerDied","Data":"2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4"}
Sep 08 20:50:21.345636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.345584    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp" event={"ID":"25eadd26-de1a-4d62-8ba7-e4e306806b80","Type":"ContainerDied","Data":"cffb79d9a7cc23070eb5f796328bcbbca1e085b222e38d90afe99ef28516cb9c"}
Sep 08 20:50:21.345636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.345619    2561 scope.go:117] "RemoveContainer" containerID="2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4"
Sep 08 20:50:21.358541 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.358501    2561 scope.go:117] "RemoveContainer" containerID="f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4"
Sep 08 20:50:21.358744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.358720    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25eadd26-de1a-4d62-8ba7-e4e306806b80-istiod-ca-cert\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.358799 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.358780    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-token\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.358868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.358854    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-socket\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.358997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.358953    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-data\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.359063 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.358996    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-podinfo\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.359357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359120    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-certs\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.359357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359228    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:21.359357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359238    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25eadd26-de1a-4d62-8ba7-e4e306806b80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:21.359357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359241    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-envoy\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.359357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359296    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsnkw\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-kube-api-access-bsnkw\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.359357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359301    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-data" (OuterVolumeSpecName: "istio-data") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:21.359357 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359330    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-credential-socket\") pod \"25eadd26-de1a-4d62-8ba7-e4e306806b80\" (UID: \"25eadd26-de1a-4d62-8ba7-e4e306806b80\") "
Sep 08 20:50:21.359769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359419    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:21.359769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359702    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:21.359769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359723    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.359769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359742    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.359769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359759    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.360060 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.359776    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25eadd26-de1a-4d62-8ba7-e4e306806b80-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.361624 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.361598    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:21.361742 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.361669    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:50:21.361842 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.361822    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-token" (OuterVolumeSpecName: "istio-token") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:21.361919 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.361850    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-kube-api-access-bsnkw" (OuterVolumeSpecName: "kube-api-access-bsnkw") pod "25eadd26-de1a-4d62-8ba7-e4e306806b80" (UID: "25eadd26-de1a-4d62-8ba7-e4e306806b80"). InnerVolumeSpecName "kube-api-access-bsnkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:21.383028 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.383005    2561 scope.go:117] "RemoveContainer" containerID="2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4"
Sep 08 20:50:21.383370 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:50:21.383348    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4\": container with ID starting with 2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4 not found: ID does not exist" containerID="2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4"
Sep 08 20:50:21.383418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.383382    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4"} err="failed to get container status \"2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4\": rpc error: code = NotFound desc = could not find container \"2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4\": container with ID starting with 2ab1f4a358562302127cac132bb53268f6f73112acacc64c1936206d6dd321e4 not found: ID does not exist"
Sep 08 20:50:21.383418 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.383409    2561 scope.go:117] "RemoveContainer" containerID="f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4"
Sep 08 20:50:21.383702 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:50:21.383678    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4\": container with ID starting with f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4 not found: ID does not exist" containerID="f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4"
Sep 08 20:50:21.383766 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.383709    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4"} err="failed to get container status \"f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4\": rpc error: code = NotFound desc = could not find container \"f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4\": container with ID starting with f08673ccde742280022b99ae631c939231df26803ac9dae108a3ea8a090e05f4 not found: ID does not exist"
Sep 08 20:50:21.460634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.460597    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.460634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.460628    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.460634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.460640    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.460874 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.460652    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsnkw\" (UniqueName: \"kubernetes.io/projected/25eadd26-de1a-4d62-8ba7-e4e306806b80-kube-api-access-bsnkw\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.460874 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.460664    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25eadd26-de1a-4d62-8ba7-e4e306806b80-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:21.670517 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.670485    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"]
Sep 08 20:50:21.673382 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:21.673360    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-792dc6-openshift-default-7444b7447b-sk2kp"]
Sep 08 20:50:22.363785 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:22.363754    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" path="/var/lib/kubelet/pods/25eadd26-de1a-4d62-8ba7-e4e306806b80/volumes"
Sep 08 20:50:27.080310 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.080262    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"]
Sep 08 20:50:27.083220 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.083194    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-27d4c1-29815005-6v96d"]
Sep 08 20:50:27.108305 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.108268    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"]
Sep 08 20:50:27.108579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.108546    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm" podUID="a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" containerName="payload-processing" containerID="cri-o://a7575beb716fb2c37f462173756a1be32583a242104b0e61a3d47d33d5f596ac" gracePeriod=30
Sep 08 20:50:27.113927 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.113868    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"]
Sep 08 20:50:27.114142 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.114118    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs" podUID="f38d5233-b15b-4507-ace3-45f6479f4584" containerName="payload-pre-processing" containerID="cri-o://e960ff300a4ce98b821cf479c49d77fa45a469277644e118a74a40bbc97f7a19" gracePeriod=30
Sep 08 20:50:27.286347 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.286298    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"]
Sep 08 20:50:27.290649 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.290623    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-27d4c1-w96z7"]
Sep 08 20:50:27.324110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.324048    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs" podUID="f38d5233-b15b-4507-ace3-45f6479f4584" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.85:9004: connect: connection refused"
Sep 08 20:50:27.390697 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.390652    2561 generic.go:356] "Generic (PLEG): container finished" podID="f38d5233-b15b-4507-ace3-45f6479f4584" containerID="e960ff300a4ce98b821cf479c49d77fa45a469277644e118a74a40bbc97f7a19" exitCode=0
Sep 08 20:50:27.390874 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.390719    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs" event={"ID":"f38d5233-b15b-4507-ace3-45f6479f4584","Type":"ContainerDied","Data":"e960ff300a4ce98b821cf479c49d77fa45a469277644e118a74a40bbc97f7a19"}
Sep 08 20:50:27.392681 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.392651    2561 generic.go:356] "Generic (PLEG): container finished" podID="a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" containerID="a7575beb716fb2c37f462173756a1be32583a242104b0e61a3d47d33d5f596ac" exitCode=0
Sep 08 20:50:27.392774 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.392732    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm" event={"ID":"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce","Type":"ContainerDied","Data":"a7575beb716fb2c37f462173756a1be32583a242104b0e61a3d47d33d5f596ac"}
Sep 08 20:50:27.420474 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.420447    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:50:27.424491 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.424463    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:50:27.524007 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.523972    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skkvv\" (UniqueName: \"kubernetes.io/projected/f38d5233-b15b-4507-ace3-45f6479f4584-kube-api-access-skkvv\") pod \"f38d5233-b15b-4507-ace3-45f6479f4584\" (UID: \"f38d5233-b15b-4507-ace3-45f6479f4584\") "
Sep 08 20:50:27.524213 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.524121    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-plugins-config-volume\") pod \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\" (UID: \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\") "
Sep 08 20:50:27.524213 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.524170    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4djhg\" (UniqueName: \"kubernetes.io/projected/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-kube-api-access-4djhg\") pod \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\" (UID: \"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce\") "
Sep 08 20:50:27.524213 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.524191    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f38d5233-b15b-4507-ace3-45f6479f4584-plugins-config-volume\") pod \"f38d5233-b15b-4507-ace3-45f6479f4584\" (UID: \"f38d5233-b15b-4507-ace3-45f6479f4584\") "
Sep 08 20:50:27.524545 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.524517    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" (UID: "a6a458b7-0de8-41d3-80a7-6cf3f491f1ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:27.524644 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.524570    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f38d5233-b15b-4507-ace3-45f6479f4584-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f38d5233-b15b-4507-ace3-45f6479f4584" (UID: "f38d5233-b15b-4507-ace3-45f6479f4584"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:27.526258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.526233    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f38d5233-b15b-4507-ace3-45f6479f4584-kube-api-access-skkvv" (OuterVolumeSpecName: "kube-api-access-skkvv") pod "f38d5233-b15b-4507-ace3-45f6479f4584" (UID: "f38d5233-b15b-4507-ace3-45f6479f4584"). InnerVolumeSpecName "kube-api-access-skkvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:27.526397 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.526383    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-kube-api-access-4djhg" (OuterVolumeSpecName: "kube-api-access-4djhg") pod "a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" (UID: "a6a458b7-0de8-41d3-80a7-6cf3f491f1ce"). InnerVolumeSpecName "kube-api-access-4djhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:27.625184 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.625091    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:27.625184 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.625124    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4djhg\" (UniqueName: \"kubernetes.io/projected/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce-kube-api-access-4djhg\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:27.625184 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.625137    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f38d5233-b15b-4507-ace3-45f6479f4584-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:27.625184 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:27.625149    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skkvv\" (UniqueName: \"kubernetes.io/projected/f38d5233-b15b-4507-ace3-45f6479f4584-kube-api-access-skkvv\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:28.325011 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.324952    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm" podUID="a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.86:9004: i/o timeout"
Sep 08 20:50:28.364434 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.364399    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fbdadce-ecad-47c5-a403-a77515109228" path="/var/lib/kubelet/pods/6fbdadce-ecad-47c5-a403-a77515109228/volumes"
Sep 08 20:50:28.364714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.364701    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9feedae1-c30d-4c00-bcff-e219a526768a" path="/var/lib/kubelet/pods/9feedae1-c30d-4c00-bcff-e219a526768a/volumes"
Sep 08 20:50:28.400097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.400019    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"
Sep 08 20:50:28.400097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.400063    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs" event={"ID":"f38d5233-b15b-4507-ace3-45f6479f4584","Type":"ContainerDied","Data":"c72e06367d899baa6663c7d5500245931565a7d2bbfb254ce2e4947a6b7c6417"}
Sep 08 20:50:28.400276 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.400128    2561 scope.go:117] "RemoveContainer" containerID="e960ff300a4ce98b821cf479c49d77fa45a469277644e118a74a40bbc97f7a19"
Sep 08 20:50:28.401552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.401534    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"
Sep 08 20:50:28.401662 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.401534    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm" event={"ID":"a6a458b7-0de8-41d3-80a7-6cf3f491f1ce","Type":"ContainerDied","Data":"196e741b770c8bc616fd833b9b0c528891158b11d725d16a8c70828b59d4fc80"}
Sep 08 20:50:28.409956 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.409936    2561 scope.go:117] "RemoveContainer" containerID="a7575beb716fb2c37f462173756a1be32583a242104b0e61a3d47d33d5f596ac"
Sep 08 20:50:28.420736 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.420716    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"]
Sep 08 20:50:28.423637 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.423616    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-27d4c1-56776cc5cf-kmxwm"]
Sep 08 20:50:28.435132 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.435105    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"]
Sep 08 20:50:28.441117 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:28.441094    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-27d4c1-84b9f6c678-qftgs"]
Sep 08 20:50:30.364178 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:30.364148    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" path="/var/lib/kubelet/pods/a6a458b7-0de8-41d3-80a7-6cf3f491f1ce/volumes"
Sep 08 20:50:30.364589 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:30.364505    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f38d5233-b15b-4507-ace3-45f6479f4584" path="/var/lib/kubelet/pods/f38d5233-b15b-4507-ace3-45f6479f4584/volumes"
Sep 08 20:50:33.902413 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:33.902378    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"]
Sep 08 20:50:33.902959 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:33.902616    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy" containerID="cri-o://a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087" gracePeriod=30
Sep 08 20:50:34.260538 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.260450    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"]
Sep 08 20:50:34.261032 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261013    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38d5233-b15b-4507-ace3-45f6479f4584" containerName="payload-pre-processing"
Sep 08 20:50:34.261140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261033    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38d5233-b15b-4507-ace3-45f6479f4584" containerName="payload-pre-processing"
Sep 08 20:50:34.261140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261059    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy"
Sep 08 20:50:34.261140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261068    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy"
Sep 08 20:50:34.261140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261078    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy"
Sep 08 20:50:34.261140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261085    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy"
Sep 08 20:50:34.261140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261110    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" containerName="payload-processing"
Sep 08 20:50:34.261140 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261119    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" containerName="payload-processing"
Sep 08 20:50:34.261480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261148    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fbdadce-ecad-47c5-a403-a77515109228" containerName="revoke-keys"
Sep 08 20:50:34.261480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261157    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fbdadce-ecad-47c5-a403-a77515109228" containerName="revoke-keys"
Sep 08 20:50:34.261480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261270    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy"
Sep 08 20:50:34.261480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261282    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f38d5233-b15b-4507-ace3-45f6479f4584" containerName="payload-pre-processing"
Sep 08 20:50:34.261480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261293    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a458b7-0de8-41d3-80a7-6cf3f491f1ce" containerName="payload-processing"
Sep 08 20:50:34.261480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261302    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fbdadce-ecad-47c5-a403-a77515109228" containerName="revoke-keys"
Sep 08 20:50:34.261480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.261314    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="25eadd26-de1a-4d62-8ba7-e4e306806b80" containerName="istio-proxy"
Sep 08 20:50:34.266174 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.266150    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"
Sep 08 20:50:34.269221 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.268992    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:50:34.272960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.272936    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"]
Sep 08 20:50:34.390230 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.390196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3721ca17-ef47-4728-a3d9-6d9e0852b0ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm\" (UID: \"3721ca17-ef47-4728-a3d9-6d9e0852b0ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"
Sep 08 20:50:34.491269 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.491233    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3721ca17-ef47-4728-a3d9-6d9e0852b0ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm\" (UID: \"3721ca17-ef47-4728-a3d9-6d9e0852b0ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"
Sep 08 20:50:34.491804 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.491784    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3721ca17-ef47-4728-a3d9-6d9e0852b0ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm\" (UID: \"3721ca17-ef47-4728-a3d9-6d9e0852b0ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"
Sep 08 20:50:34.581455 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.581367    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"
Sep 08 20:50:34.858429 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:34.858403    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"]
Sep 08 20:50:34.860877 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:50:34.860844    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3721ca17_ef47_4728_a3d9_6d9e0852b0ed.slice/crio-75845e7e2c0eec207d05694dd49190de7aa2b70dc8e78146c15c804827d9f225 WatchSource:0}: Error finding container 75845e7e2c0eec207d05694dd49190de7aa2b70dc8e78146c15c804827d9f225: Status 404 returned error can't find the container with id 75845e7e2c0eec207d05694dd49190de7aa2b70dc8e78146c15c804827d9f225
Sep 08 20:50:35.456405 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:35.456371    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm" event={"ID":"3721ca17-ef47-4728-a3d9-6d9e0852b0ed","Type":"ContainerStarted","Data":"3985e6bb78734442ba645b56cbaceb5ccc4886ab0a3a953e8136f301a553053e"}
Sep 08 20:50:35.456800 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:35.456428    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm" event={"ID":"3721ca17-ef47-4728-a3d9-6d9e0852b0ed","Type":"ContainerStarted","Data":"75845e7e2c0eec207d05694dd49190de7aa2b70dc8e78146c15c804827d9f225"}
Sep 08 20:50:35.476040 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:35.475978    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm" podStartSLOduration=1.475960891 podStartE2EDuration="1.475960891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:50:35.472016432 +0000 UTC m=+1521.637187009" watchObservedRunningTime="2026-09-08 20:50:35.475960891 +0000 UTC m=+1521.641131515"
Sep 08 20:50:36.465263 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:36.465178    2561 generic.go:356] "Generic (PLEG): container finished" podID="3721ca17-ef47-4728-a3d9-6d9e0852b0ed" containerID="3985e6bb78734442ba645b56cbaceb5ccc4886ab0a3a953e8136f301a553053e" exitCode=0
Sep 08 20:50:36.465684 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:36.465268    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm" event={"ID":"3721ca17-ef47-4728-a3d9-6d9e0852b0ed","Type":"ContainerDied","Data":"3985e6bb78734442ba645b56cbaceb5ccc4886ab0a3a953e8136f301a553053e"}
Sep 08 20:50:37.620537 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:37.620510    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"
Sep 08 20:50:37.724706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:37.724609    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3721ca17-ef47-4728-a3d9-6d9e0852b0ed-maas-api-service-ca\") pod \"3721ca17-ef47-4728-a3d9-6d9e0852b0ed\" (UID: \"3721ca17-ef47-4728-a3d9-6d9e0852b0ed\") "
Sep 08 20:50:37.725029 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:37.725006    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3721ca17-ef47-4728-a3d9-6d9e0852b0ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3721ca17-ef47-4728-a3d9-6d9e0852b0ed" (UID: "3721ca17-ef47-4728-a3d9-6d9e0852b0ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:37.826188 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:37.826148    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3721ca17-ef47-4728-a3d9-6d9e0852b0ed-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:38.481090 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:38.481059    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"
Sep 08 20:50:38.481260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:38.481114    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm" event={"ID":"3721ca17-ef47-4728-a3d9-6d9e0852b0ed","Type":"ContainerDied","Data":"75845e7e2c0eec207d05694dd49190de7aa2b70dc8e78146c15c804827d9f225"}
Sep 08 20:50:38.481260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:38.481163    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75845e7e2c0eec207d05694dd49190de7aa2b70dc8e78146c15c804827d9f225"
Sep 08 20:50:39.179022 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.178953    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:50:39.240398 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240355    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd571727-7e02-4cca-8725-6d7a3489bf68-istiod-ca-cert\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.240398 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240405    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-certs\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240435    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-credential-socket\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240487    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-envoy\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240511    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stw9j\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-kube-api-access-stw9j\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240552    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-token\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240576    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-data\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240646    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-podinfo\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240695    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-socket\") pod \"cd571727-7e02-4cca-8725-6d7a3489bf68\" (UID: \"cd571727-7e02-4cca-8725-6d7a3489bf68\") "
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240826    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.240868    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd571727-7e02-4cca-8725-6d7a3489bf68-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.241097    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.241112    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.241173    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd571727-7e02-4cca-8725-6d7a3489bf68-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.241414 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.241191    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.242135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.241465    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:39.243408 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.243363    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:39.243542 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.243496    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-kube-api-access-stw9j" (OuterVolumeSpecName: "kube-api-access-stw9j") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "kube-api-access-stw9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:39.243607 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.243580    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:39.243810 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.243783    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd571727-7e02-4cca-8725-6d7a3489bf68" (UID: "cd571727-7e02-4cca-8725-6d7a3489bf68"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:50:39.342286 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.342247    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.342286 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.342279    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.342286 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.342289    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.342286 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.342298    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stw9j\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-kube-api-access-stw9j\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.342562 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.342309    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.342562 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.342317    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.342562 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.342324    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd571727-7e02-4cca-8725-6d7a3489bf68-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:39.490017 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.489927    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerID="a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087" exitCode=0
Sep 08 20:50:39.490017 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.489981    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"
Sep 08 20:50:39.490210 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.490017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" event={"ID":"cd571727-7e02-4cca-8725-6d7a3489bf68","Type":"ContainerDied","Data":"a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087"}
Sep 08 20:50:39.490210 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.490070    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw" event={"ID":"cd571727-7e02-4cca-8725-6d7a3489bf68","Type":"ContainerDied","Data":"779393c6b9679008ac8aedf0385a65d0a97b2384449295800434d3ac1951c192"}
Sep 08 20:50:39.490210 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.490092    2561 scope.go:117] "RemoveContainer" containerID="a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087"
Sep 08 20:50:39.500866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.500845    2561 scope.go:117] "RemoveContainer" containerID="7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec"
Sep 08 20:50:39.510856 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.510824    2561 scope.go:117] "RemoveContainer" containerID="a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087"
Sep 08 20:50:39.511212 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:50:39.511194    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087\": container with ID starting with a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087 not found: ID does not exist" containerID="a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087"
Sep 08 20:50:39.511260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.511221    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087"} err="failed to get container status \"a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087\": rpc error: code = NotFound desc = could not find container \"a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087\": container with ID starting with a1f8d51a8b88249f0ced75269f8f54c7bfdac39de246d6105858e70a9299c087 not found: ID does not exist"
Sep 08 20:50:39.511260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.511238    2561 scope.go:117] "RemoveContainer" containerID="7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec"
Sep 08 20:50:39.511474 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:50:39.511457    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec\": container with ID starting with 7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec not found: ID does not exist" containerID="7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec"
Sep 08 20:50:39.511514 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.511479    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec"} err="failed to get container status \"7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec\": rpc error: code = NotFound desc = could not find container \"7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec\": container with ID starting with 7d849d71043fd53499b83e8706003197d69e4de7981d9e0f602bb719f20e22ec not found: ID does not exist"
Sep 08 20:50:39.526866 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.526834    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"]
Sep 08 20:50:39.534544 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:39.534512    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-27d4c1-openshift-default-5f95b4c8bd-dhvmw"]
Sep 08 20:50:40.366759 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:40.366726    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" path="/var/lib/kubelet/pods/cd571727-7e02-4cca-8725-6d7a3489bf68/volumes"
Sep 08 20:50:44.880700 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:44.880660    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"]
Sep 08 20:50:44.881517 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:44.881478    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr" podUID="9564f612-bac0-4c19-ba35-a882800c253c" containerName="payload-processing" containerID="cri-o://9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7" gracePeriod=30
Sep 08 20:50:44.885573 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:44.885529    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"]
Sep 08 20:50:44.885784 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:44.885756    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg" podUID="cca8bf38-44d5-491a-83be-adfbbd61600d" containerName="payload-pre-processing" containerID="cri-o://1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119" gracePeriod=30
Sep 08 20:50:45.035098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.035048    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"]
Sep 08 20:50:45.040131 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.040093    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-74f9a1-t6hcm"]
Sep 08 20:50:45.193453 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.193421    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:50:45.199751 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.199720    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:50:45.298532 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.298491    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2gh8\" (UniqueName: \"kubernetes.io/projected/cca8bf38-44d5-491a-83be-adfbbd61600d-kube-api-access-v2gh8\") pod \"cca8bf38-44d5-491a-83be-adfbbd61600d\" (UID: \"cca8bf38-44d5-491a-83be-adfbbd61600d\") "
Sep 08 20:50:45.298714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.298561    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9564f612-bac0-4c19-ba35-a882800c253c-plugins-config-volume\") pod \"9564f612-bac0-4c19-ba35-a882800c253c\" (UID: \"9564f612-bac0-4c19-ba35-a882800c253c\") "
Sep 08 20:50:45.298714 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.298586    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkxn2\" (UniqueName: \"kubernetes.io/projected/9564f612-bac0-4c19-ba35-a882800c253c-kube-api-access-kkxn2\") pod \"9564f612-bac0-4c19-ba35-a882800c253c\" (UID: \"9564f612-bac0-4c19-ba35-a882800c253c\") "
Sep 08 20:50:45.298791 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.298715    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cca8bf38-44d5-491a-83be-adfbbd61600d-plugins-config-volume\") pod \"cca8bf38-44d5-491a-83be-adfbbd61600d\" (UID: \"cca8bf38-44d5-491a-83be-adfbbd61600d\") "
Sep 08 20:50:45.299042 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.299017    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9564f612-bac0-4c19-ba35-a882800c253c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9564f612-bac0-4c19-ba35-a882800c253c" (UID: "9564f612-bac0-4c19-ba35-a882800c253c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:45.299130 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.299102    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cca8bf38-44d5-491a-83be-adfbbd61600d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cca8bf38-44d5-491a-83be-adfbbd61600d" (UID: "cca8bf38-44d5-491a-83be-adfbbd61600d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:45.299130 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.299123    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9564f612-bac0-4c19-ba35-a882800c253c-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:45.300777 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.300748    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca8bf38-44d5-491a-83be-adfbbd61600d-kube-api-access-v2gh8" (OuterVolumeSpecName: "kube-api-access-v2gh8") pod "cca8bf38-44d5-491a-83be-adfbbd61600d" (UID: "cca8bf38-44d5-491a-83be-adfbbd61600d"). InnerVolumeSpecName "kube-api-access-v2gh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:45.300872 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.300793    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9564f612-bac0-4c19-ba35-a882800c253c-kube-api-access-kkxn2" (OuterVolumeSpecName: "kube-api-access-kkxn2") pod "9564f612-bac0-4c19-ba35-a882800c253c" (UID: "9564f612-bac0-4c19-ba35-a882800c253c"). InnerVolumeSpecName "kube-api-access-kkxn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:45.400367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.400321    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2gh8\" (UniqueName: \"kubernetes.io/projected/cca8bf38-44d5-491a-83be-adfbbd61600d-kube-api-access-v2gh8\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:45.400367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.400356    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkxn2\" (UniqueName: \"kubernetes.io/projected/9564f612-bac0-4c19-ba35-a882800c253c-kube-api-access-kkxn2\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:45.400367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.400366    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cca8bf38-44d5-491a-83be-adfbbd61600d-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:50:45.544945 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.544841    2561 generic.go:356] "Generic (PLEG): container finished" podID="9564f612-bac0-4c19-ba35-a882800c253c" containerID="9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7" exitCode=0
Sep 08 20:50:45.544945 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.544889    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"
Sep 08 20:50:45.544945 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.544933    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr" event={"ID":"9564f612-bac0-4c19-ba35-a882800c253c","Type":"ContainerDied","Data":"9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7"}
Sep 08 20:50:45.545247 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.544990    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr" event={"ID":"9564f612-bac0-4c19-ba35-a882800c253c","Type":"ContainerDied","Data":"9440fb7da9562ad502512b08feb464078a2dcb0d13382256a2ff63fe27f141d3"}
Sep 08 20:50:45.545247 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.545024    2561 scope.go:117] "RemoveContainer" containerID="9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7"
Sep 08 20:50:45.546815 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.546787    2561 generic.go:356] "Generic (PLEG): container finished" podID="cca8bf38-44d5-491a-83be-adfbbd61600d" containerID="1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119" exitCode=0
Sep 08 20:50:45.546956 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.546848    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"
Sep 08 20:50:45.546956 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.546872    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg" event={"ID":"cca8bf38-44d5-491a-83be-adfbbd61600d","Type":"ContainerDied","Data":"1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119"}
Sep 08 20:50:45.546956 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.546944    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg" event={"ID":"cca8bf38-44d5-491a-83be-adfbbd61600d","Type":"ContainerDied","Data":"21c582f5f9e0b97c36c526f0aed9048f0ae505751ee95b5fe049bb8b68862215"}
Sep 08 20:50:45.556281 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.556263    2561 scope.go:117] "RemoveContainer" containerID="9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7"
Sep 08 20:50:45.556674 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:50:45.556646    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7\": container with ID starting with 9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7 not found: ID does not exist" containerID="9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7"
Sep 08 20:50:45.556738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.556689    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7"} err="failed to get container status \"9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7\": rpc error: code = NotFound desc = could not find container \"9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7\": container with ID starting with 9c2068c4dad74bbc02bdde03967d2e875971d1c685bfee97c668250ff0c16ff7 not found: ID does not exist"
Sep 08 20:50:45.556738 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.556715    2561 scope.go:117] "RemoveContainer" containerID="1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119"
Sep 08 20:50:45.566829 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.566807    2561 scope.go:117] "RemoveContainer" containerID="1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119"
Sep 08 20:50:45.567182 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:50:45.567164    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119\": container with ID starting with 1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119 not found: ID does not exist" containerID="1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119"
Sep 08 20:50:45.567264 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.567188    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119"} err="failed to get container status \"1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119\": rpc error: code = NotFound desc = could not find container \"1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119\": container with ID starting with 1a62c4e58031a4d1b409bbc104998d62a83f58b9931a793a648c914918525119 not found: ID does not exist"
Sep 08 20:50:45.571947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.571910    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"]
Sep 08 20:50:45.574662 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.574634    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-74f9a1-759f9f95-b4gjr"]
Sep 08 20:50:45.588096 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.588058    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"]
Sep 08 20:50:45.598302 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:45.594325    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-74f9a1-85b7699898-dbtfg"]
Sep 08 20:50:46.364803 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:46.364767    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3721ca17-ef47-4728-a3d9-6d9e0852b0ed" path="/var/lib/kubelet/pods/3721ca17-ef47-4728-a3d9-6d9e0852b0ed/volumes"
Sep 08 20:50:46.365208 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:46.365167    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9564f612-bac0-4c19-ba35-a882800c253c" path="/var/lib/kubelet/pods/9564f612-bac0-4c19-ba35-a882800c253c/volumes"
Sep 08 20:50:46.365493 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:50:46.365479    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cca8bf38-44d5-491a-83be-adfbbd61600d" path="/var/lib/kubelet/pods/cca8bf38-44d5-491a-83be-adfbbd61600d/volumes"
Sep 08 20:51:04.316055 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316021    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"]
Sep 08 20:51:04.316479 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316462    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy"
Sep 08 20:51:04.316479 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316479    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy"
Sep 08 20:51:04.316560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316495    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cca8bf38-44d5-491a-83be-adfbbd61600d" containerName="payload-pre-processing"
Sep 08 20:51:04.316560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316501    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cca8bf38-44d5-491a-83be-adfbbd61600d" containerName="payload-pre-processing"
Sep 08 20:51:04.316560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316507    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9564f612-bac0-4c19-ba35-a882800c253c" containerName="payload-processing"
Sep 08 20:51:04.316560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316513    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9564f612-bac0-4c19-ba35-a882800c253c" containerName="payload-processing"
Sep 08 20:51:04.316560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316524    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3721ca17-ef47-4728-a3d9-6d9e0852b0ed" containerName="revoke-keys"
Sep 08 20:51:04.316560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316530    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3721ca17-ef47-4728-a3d9-6d9e0852b0ed" containerName="revoke-keys"
Sep 08 20:51:04.316747 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316590    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cca8bf38-44d5-491a-83be-adfbbd61600d" containerName="payload-pre-processing"
Sep 08 20:51:04.316747 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316599    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3721ca17-ef47-4728-a3d9-6d9e0852b0ed" containerName="revoke-keys"
Sep 08 20:51:04.316747 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316606    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9564f612-bac0-4c19-ba35-a882800c253c" containerName="payload-processing"
Sep 08 20:51:04.316747 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316613    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy"
Sep 08 20:51:04.316747 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.316619    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy"
Sep 08 20:51:04.321527 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.321506    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.324300 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.324276    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-aab326-openshift-default-dockercfg-kjngc\""
Sep 08 20:51:04.339994 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.339965    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"]
Sep 08 20:51:04.487936 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.487871    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-envoy\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.488111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.487988    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-credential-socket\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.488111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.488026    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-data\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.488111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.488057    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-token\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.488111 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.488075    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4009667b-5916-45cc-b2f6-ffe08e05b731-istiod-ca-cert\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.488261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.488132    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-podinfo\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.488261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.488171    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-certs\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.488261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.488193    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsrjg\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-kube-api-access-tsrjg\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.488261 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.488240    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-socket\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589472 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589367    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-envoy\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589472 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589431    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-credential-socket\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589719 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589488    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-data\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589719 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589523    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-token\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589719 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4009667b-5916-45cc-b2f6-ffe08e05b731-istiod-ca-cert\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589719 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589690    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-podinfo\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589990 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589776    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-certs\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589990 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsrjg\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-kube-api-access-tsrjg\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589990 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589815    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-credential-socket\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589990 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-socket\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.589990 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.589939    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-data\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.590263 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.590124    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-certs\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.590322 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.590305    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-socket\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.590373 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.590321    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4009667b-5916-45cc-b2f6-ffe08e05b731-istiod-ca-cert\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.591936 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.591915    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-envoy\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.592067 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.592048    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-podinfo\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.597241 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.597218    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-token\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.597369 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.597338    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsrjg\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-kube-api-access-tsrjg\") pod \"e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") " pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.632473 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.632433    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:04.917135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.917096    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"]
Sep 08 20:51:04.918695 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:51:04.918658    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4009667b_5916_45cc_b2f6_ffe08e05b731.slice/crio-0307386ec1de204a617aa99663f86a853749a5e1dfd25eaddc22570677909eca WatchSource:0}: Error finding container 0307386ec1de204a617aa99663f86a853749a5e1dfd25eaddc22570677909eca: Status 404 returned error can't find the container with id 0307386ec1de204a617aa99663f86a853749a5e1dfd25eaddc22570677909eca
Sep 08 20:51:04.921115 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.921080    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:04.921194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.921149    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:04.921194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:04.921177    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:05.698940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:05.698886    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" event={"ID":"4009667b-5916-45cc-b2f6-ffe08e05b731","Type":"ContainerStarted","Data":"d683dba9995fba1ac195d4c960ddcf4a84468223ea4ed389220213a6572b0d82"}
Sep 08 20:51:05.699352 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:05.698967    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" event={"ID":"4009667b-5916-45cc-b2f6-ffe08e05b731","Type":"ContainerStarted","Data":"0307386ec1de204a617aa99663f86a853749a5e1dfd25eaddc22570677909eca"}
Sep 08 20:51:05.722307 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:05.722245    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" podStartSLOduration=1.722228711 podStartE2EDuration="1.722228711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:51:05.720289647 +0000 UTC m=+1551.885460236" watchObservedRunningTime="2026-09-08 20:51:05.722228711 +0000 UTC m=+1551.887399287"
Sep 08 20:51:06.633560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:06.633515    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:06.640108 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:06.640079    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:06.705766 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:06.705731    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:06.706766 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:06.706745    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:09.376748 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.376713    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"]
Sep 08 20:51:09.377262 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.377245    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy"
Sep 08 20:51:09.377304 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.377264    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd571727-7e02-4cca-8725-6d7a3489bf68" containerName="istio-proxy"
Sep 08 20:51:09.381664 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.381641    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:09.385290 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.385255    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-aab326-dockercfg-bj6g8\""
Sep 08 20:51:09.385961 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.385939    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-aab326\""
Sep 08 20:51:09.395742 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.395719    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"]
Sep 08 20:51:09.539730 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.539690    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmk6k\" (UniqueName: \"kubernetes.io/projected/3cbe9fa6-6645-4f3a-85e9-838dff529487-kube-api-access-jmk6k\") pod \"payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b\" (UID: \"3cbe9fa6-6645-4f3a-85e9-838dff529487\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:09.539945 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.539875    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cbe9fa6-6645-4f3a-85e9-838dff529487-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b\" (UID: \"3cbe9fa6-6645-4f3a-85e9-838dff529487\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:09.641152 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.641038    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cbe9fa6-6645-4f3a-85e9-838dff529487-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b\" (UID: \"3cbe9fa6-6645-4f3a-85e9-838dff529487\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:09.641152 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.641125    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmk6k\" (UniqueName: \"kubernetes.io/projected/3cbe9fa6-6645-4f3a-85e9-838dff529487-kube-api-access-jmk6k\") pod \"payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b\" (UID: \"3cbe9fa6-6645-4f3a-85e9-838dff529487\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:09.641768 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.641742    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cbe9fa6-6645-4f3a-85e9-838dff529487-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b\" (UID: \"3cbe9fa6-6645-4f3a-85e9-838dff529487\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:09.650075 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.650047    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmk6k\" (UniqueName: \"kubernetes.io/projected/3cbe9fa6-6645-4f3a-85e9-838dff529487-kube-api-access-jmk6k\") pod \"payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b\" (UID: \"3cbe9fa6-6645-4f3a-85e9-838dff529487\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:09.696362 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.696331    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:09.973647 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:09.973617    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"]
Sep 08 20:51:09.974876 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:51:09.974849    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cbe9fa6_6645_4f3a_85e9_838dff529487.slice/crio-0cf692454a49eb0aa2760f95b62b967ccecfded76c80ec958e7c647b9da7f61c WatchSource:0}: Error finding container 0cf692454a49eb0aa2760f95b62b967ccecfded76c80ec958e7c647b9da7f61c: Status 404 returned error can't find the container with id 0cf692454a49eb0aa2760f95b62b967ccecfded76c80ec958e7c647b9da7f61c
Sep 08 20:51:10.736317 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:10.736278    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b" event={"ID":"3cbe9fa6-6645-4f3a-85e9-838dff529487","Type":"ContainerStarted","Data":"877953ebdf5373304385e612beeea81d23610855193c7f2c0c055faccbcf1a70"}
Sep 08 20:51:10.736817 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:10.736342    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b" event={"ID":"3cbe9fa6-6645-4f3a-85e9-838dff529487","Type":"ContainerStarted","Data":"0cf692454a49eb0aa2760f95b62b967ccecfded76c80ec958e7c647b9da7f61c"}
Sep 08 20:51:10.736817 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:10.736475    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:10.754449 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:10.754399    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b" podStartSLOduration=1.7543838200000001 podStartE2EDuration="1.75438382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:51:10.751653957 +0000 UTC m=+1556.916824525" watchObservedRunningTime="2026-09-08 20:51:10.75438382 +0000 UTC m=+1556.919554434"
Sep 08 20:51:14.827725 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:14.827694    2561 scope.go:117] "RemoveContainer" containerID="3dff9967405d41192ed6b1d8215faf0376e969ff0de120bdebfda8f733baee08"
Sep 08 20:51:14.839936 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:14.839875    2561 scope.go:117] "RemoveContainer" containerID="3f2a2aed96ac18a9c0fa296af4e9b59b2853f4172e3fc051b0c3b42ff8a7a1dc"
Sep 08 20:51:14.850032 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:14.850010    2561 scope.go:117] "RemoveContainer" containerID="c8c33326872e0b97f29028b13628f4b1f471ef62a8c90ba1adc0d8d43ab9a2a1"
Sep 08 20:51:21.745335 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:21.745293    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:51:22.748080 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:22.748034    2561 patch_prober.go:28] interesting pod/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 20:51:22.748671 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:22.748112    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 20:51:24.840106 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:24.840019    2561 generic.go:356] "Generic (PLEG): container finished" podID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerID="d683dba9995fba1ac195d4c960ddcf4a84468223ea4ed389220213a6572b0d82" exitCode=0
Sep 08 20:51:24.840106 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:24.840058    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" event={"ID":"4009667b-5916-45cc-b2f6-ffe08e05b731","Type":"ContainerDied","Data":"d683dba9995fba1ac195d4c960ddcf4a84468223ea4ed389220213a6572b0d82"}
Sep 08 20:51:24.840622 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:24.840575    2561 scope.go:117] "RemoveContainer" containerID="d683dba9995fba1ac195d4c960ddcf4a84468223ea4ed389220213a6572b0d82"
Sep 08 20:51:24.841911 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:24.841865    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:24.842039 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:24.841994    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:24.842113 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:24.842043    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:25.853882 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:25.853843    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" event={"ID":"4009667b-5916-45cc-b2f6-ffe08e05b731","Type":"ContainerStarted","Data":"d506755062f447b362ac5a6f8986ac765b7f8e0f11d766d0fe9a910c605f4e59"}
Sep 08 20:51:26.632993 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:26.632876    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:26.633271 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:26.633241    2561 patch_prober.go:28] interesting pod/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Sep 08 20:51:26.633505 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:26.633474    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Sep 08 20:51:27.633835 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:27.633791    2561 patch_prober.go:28] interesting pod/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Sep 08 20:51:27.634365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:27.633878    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Sep 08 20:51:28.632989 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:28.632946    2561 patch_prober.go:28] interesting pod/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Sep 08 20:51:28.633208 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:28.633034    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Sep 08 20:51:29.633302 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:29.633255    2561 patch_prober.go:28] interesting pod/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Sep 08 20:51:29.633843 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:29.633343    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Sep 08 20:51:30.633853 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:30.633808    2561 patch_prober.go:28] interesting pod/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Sep 08 20:51:30.634333 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:30.633883    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Sep 08 20:51:31.637277 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:31.637248    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:31.637725 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:31.637599    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:31.638281 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:31.638259    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:51:33.672962 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.672920    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"]
Sep 08 20:51:33.676412 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.676387    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.678867 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.678843    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-0566ff-openshift-default-dockercfg-6n7kd\""
Sep 08 20:51:33.688648 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.688621    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"]
Sep 08 20:51:33.770870 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.770830    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-data\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.771097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.770873    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-socket\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.771097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.770926    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-token\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.771097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.770998    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-credential-socket\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.771097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.771035    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-certs\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.771097 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.771078    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtc2h\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-kube-api-access-rtc2h\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.771345 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.771165    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-envoy\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.771345 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.771218    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-podinfo\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.771345 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.771265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5f2435a-d50c-484a-a9ca-b426bf19009d-istiod-ca-cert\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.872782 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.872741    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-certs\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.872782 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.872787    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtc2h\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-kube-api-access-rtc2h\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873104 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.872873    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-envoy\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873104 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.872948    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-podinfo\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873104 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.872975    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5f2435a-d50c-484a-a9ca-b426bf19009d-istiod-ca-cert\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873104 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873056    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-data\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873104 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873087    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-socket\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873362 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873128    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-token\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873362 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873159    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-credential-socket\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873362 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873254    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-certs\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873556 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873461    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-data\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873556 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-socket\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873658 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873578    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-credential-socket\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.873832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.873812    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5f2435a-d50c-484a-a9ca-b426bf19009d-istiod-ca-cert\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.875462 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.875434    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-envoy\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.875679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.875659    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-podinfo\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.881044 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.881021    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtc2h\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-kube-api-access-rtc2h\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.881160 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.881090    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-token\") pod \"e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") " pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:33.989732 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:33.989628    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:34.273211 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:34.271761    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"]
Sep 08 20:51:34.278124 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:51:34.278091    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f2435a_d50c_484a_a9ca_b426bf19009d.slice/crio-b682c00b4a4074b83e04eec396483305b8b57902f04ed2929d75e412118d061d WatchSource:0}: Error finding container b682c00b4a4074b83e04eec396483305b8b57902f04ed2929d75e412118d061d: Status 404 returned error can't find the container with id b682c00b4a4074b83e04eec396483305b8b57902f04ed2929d75e412118d061d
Sep 08 20:51:34.280533 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:34.280498    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:34.280625 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:34.280568    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:34.280625 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:34.280596    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:34.926168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:34.926126    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" event={"ID":"f5f2435a-d50c-484a-a9ca-b426bf19009d","Type":"ContainerStarted","Data":"72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c"}
Sep 08 20:51:34.926648 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:34.926186    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" event={"ID":"f5f2435a-d50c-484a-a9ca-b426bf19009d","Type":"ContainerStarted","Data":"b682c00b4a4074b83e04eec396483305b8b57902f04ed2929d75e412118d061d"}
Sep 08 20:51:34.953827 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:34.953769    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" podStartSLOduration=1.9537457900000001 podStartE2EDuration="1.95374579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:51:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:51:34.94850219 +0000 UTC m=+1581.113672767" watchObservedRunningTime="2026-09-08 20:51:34.95374579 +0000 UTC m=+1581.118916367"
Sep 08 20:51:34.990655 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:34.990610    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:35.996312 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:35.996286    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:36.940970 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:36.940942    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:36.942062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:36.942039    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:38.747743 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.747703    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"]
Sep 08 20:51:38.750592 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.750567    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:38.753516 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.753493    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-0566ff-dockercfg-zdkpv\""
Sep 08 20:51:38.753665 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.753497    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0566ff\""
Sep 08 20:51:38.761438 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.761408    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"]
Sep 08 20:51:38.782194 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.782161    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"]
Sep 08 20:51:38.784773 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.784756    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:38.797556 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.797530    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"]
Sep 08 20:51:38.826077 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.826026    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2533658-613b-4b9f-817e-5306e3525b44-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx\" (UID: \"c2533658-613b-4b9f-817e-5306e3525b44\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:38.826275 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.826144    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfzpn\" (UniqueName: \"kubernetes.io/projected/c2533658-613b-4b9f-817e-5306e3525b44-kube-api-access-wfzpn\") pod \"payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx\" (UID: \"c2533658-613b-4b9f-817e-5306e3525b44\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:38.927818 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.927773    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg\" (UID: \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:38.928061 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.927833    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2533658-613b-4b9f-817e-5306e3525b44-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx\" (UID: \"c2533658-613b-4b9f-817e-5306e3525b44\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:38.928061 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.927939    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfzpn\" (UniqueName: \"kubernetes.io/projected/c2533658-613b-4b9f-817e-5306e3525b44-kube-api-access-wfzpn\") pod \"payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx\" (UID: \"c2533658-613b-4b9f-817e-5306e3525b44\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:38.928061 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.927993    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrsr4\" (UniqueName: \"kubernetes.io/projected/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-kube-api-access-zrsr4\") pod \"payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg\" (UID: \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:38.928587 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.928571    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2533658-613b-4b9f-817e-5306e3525b44-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx\" (UID: \"c2533658-613b-4b9f-817e-5306e3525b44\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:38.937036 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:38.937006    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfzpn\" (UniqueName: \"kubernetes.io/projected/c2533658-613b-4b9f-817e-5306e3525b44-kube-api-access-wfzpn\") pod \"payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx\" (UID: \"c2533658-613b-4b9f-817e-5306e3525b44\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:39.029044 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.028954    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrsr4\" (UniqueName: \"kubernetes.io/projected/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-kube-api-access-zrsr4\") pod \"payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg\" (UID: \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:39.029234 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.029067    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg\" (UID: \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:39.029809 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.029784    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg\" (UID: \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:39.037404 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.037365    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrsr4\" (UniqueName: \"kubernetes.io/projected/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-kube-api-access-zrsr4\") pod \"payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg\" (UID: \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:39.064023 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.063986    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:39.096204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.096169    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:39.339530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.339500    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"]
Sep 08 20:51:39.341925 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:51:39.341883    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2533658_613b_4b9f_817e_5306e3525b44.slice/crio-6b6a976a776f5290c5317ea0bb217057da390e57b16c7488a9565788333c9b83 WatchSource:0}: Error finding container 6b6a976a776f5290c5317ea0bb217057da390e57b16c7488a9565788333c9b83: Status 404 returned error can't find the container with id 6b6a976a776f5290c5317ea0bb217057da390e57b16c7488a9565788333c9b83
Sep 08 20:51:39.384138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.384112    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"]
Sep 08 20:51:39.384887 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:51:39.384863    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb4fa70_e792_42f4_9a6d_c176c5b2ea17.slice/crio-279fc070df86ccd8b430c07c9de834b2bf943a248f1111b40927e991042c0533 WatchSource:0}: Error finding container 279fc070df86ccd8b430c07c9de834b2bf943a248f1111b40927e991042c0533: Status 404 returned error can't find the container with id 279fc070df86ccd8b430c07c9de834b2bf943a248f1111b40927e991042c0533
Sep 08 20:51:39.976733 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.976695    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx" event={"ID":"c2533658-613b-4b9f-817e-5306e3525b44","Type":"ContainerStarted","Data":"89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb"}
Sep 08 20:51:39.977270 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.976770    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:39.977270 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.976795    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx" event={"ID":"c2533658-613b-4b9f-817e-5306e3525b44","Type":"ContainerStarted","Data":"6b6a976a776f5290c5317ea0bb217057da390e57b16c7488a9565788333c9b83"}
Sep 08 20:51:39.978634 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.978606    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg" event={"ID":"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17","Type":"ContainerStarted","Data":"d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18"}
Sep 08 20:51:39.978744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.978660    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg" event={"ID":"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17","Type":"ContainerStarted","Data":"279fc070df86ccd8b430c07c9de834b2bf943a248f1111b40927e991042c0533"}
Sep 08 20:51:39.978744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.978689    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:39.995048 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:39.995000    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx" podStartSLOduration=1.994983393 podStartE2EDuration="1.994983393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:51:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:51:39.993288876 +0000 UTC m=+1586.158459453" watchObservedRunningTime="2026-09-08 20:51:39.994983393 +0000 UTC m=+1586.160153966"
Sep 08 20:51:40.009966 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:40.009890    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg" podStartSLOduration=2.009875696 podStartE2EDuration="2.009875696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:51:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:51:40.008577446 +0000 UTC m=+1586.173748027" watchObservedRunningTime="2026-09-08 20:51:40.009875696 +0000 UTC m=+1586.175046272"
Sep 08 20:51:50.987408 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:50.987378    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:51:50.987858 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:50.987446    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:51:52.983129 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:52.983087    2561 patch_prober.go:28] interesting pod/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 20:51:52.983582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:52.983165    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 20:51:55.108667 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:55.108580    2561 generic.go:356] "Generic (PLEG): container finished" podID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerID="72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c" exitCode=0
Sep 08 20:51:55.108667 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:55.108655    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" event={"ID":"f5f2435a-d50c-484a-a9ca-b426bf19009d","Type":"ContainerDied","Data":"72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c"}
Sep 08 20:51:55.109163 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:55.109146    2561 scope.go:117] "RemoveContainer" containerID="72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c"
Sep 08 20:51:55.110288 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:55.110256    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:55.110387 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:55.110326    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:55.110387 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:55.110358    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:51:56.117168 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:56.117134    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" event={"ID":"f5f2435a-d50c-484a-a9ca-b426bf19009d","Type":"ContainerStarted","Data":"3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796"}
Sep 08 20:51:56.990022 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:56.989976    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:51:56.990350 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:56.990324    2561 patch_prober.go:28] interesting pod/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Sep 08 20:51:56.990455 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:56.990419    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Sep 08 20:51:57.990435 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:57.990391    2561 patch_prober.go:28] interesting pod/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Sep 08 20:51:57.990844 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:57.990452    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Sep 08 20:51:58.990152 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:58.990108    2561 patch_prober.go:28] interesting pod/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Sep 08 20:51:58.990371 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:58.990185    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Sep 08 20:51:59.990655 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:59.990618    2561 patch_prober.go:28] interesting pod/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Sep 08 20:51:59.991087 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:51:59.990673    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Sep 08 20:52:00.990124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:00.990087    2561 patch_prober.go:28] interesting pod/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Sep 08 20:52:00.990299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:00.990139    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Sep 08 20:52:01.994849 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:01.994819    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:52:01.995333 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:01.995126    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:52:01.996105 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:01.996081    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:52:07.112570 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:07.112482    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mw9v6_e7b6128d-c751-4948-8312-f0ed78e55355/payload-processing/0.log"
Sep 08 20:52:07.228797 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:07.228766    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx_c2533658-613b-4b9f-817e-5306e3525b44/payload-processing/0.log"
Sep 08 20:52:14.916687 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:14.916654    2561 scope.go:117] "RemoveContainer" containerID="503cf50a63df6e1d9078f22819acf21dff768079cd3a48d483b382b367ffeb38"
Sep 08 20:52:14.926171 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:14.926149    2561 scope.go:117] "RemoveContainer" containerID="aeb3cb7887fd2994d7270448afff52ad486f7f6df15e4f5b25df9a33ebff867d"
Sep 08 20:52:42.902663 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:42.902631    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mw9v6_e7b6128d-c751-4948-8312-f0ed78e55355/payload-processing/0.log"
Sep 08 20:52:43.020624 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.020594    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx_c2533658-613b-4b9f-817e-5306e3525b44/payload-processing/0.log"
Sep 08 20:52:43.592860 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.592823    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"]
Sep 08 20:52:43.596546 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.596528    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.601688 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.601662    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-4126a9-openshift-default-dockercfg-r9z7j\""
Sep 08 20:52:43.629607 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.629571    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"]
Sep 08 20:52:43.719989 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.719925    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.719989 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.719984    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-token\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.720216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.720044    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-data\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.720216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.720061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-podinfo\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.720216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.720099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c27gh\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-kube-api-access-c27gh\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.720216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.720135    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-certs\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.720216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.720183    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-envoy\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.720384 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.720220    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-credential-socket\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.720384 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.720239    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-socket\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821350 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821316    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-envoy\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821568 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821376    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-credential-socket\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821568 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-socket\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821652    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-token\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821700    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-data\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821725    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-podinfo\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821755    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-credential-socket\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821826    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-socket\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.821868 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821858    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c27gh\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-kube-api-access-c27gh\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.822122 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.821930    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-certs\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.822185 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.822125    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-data\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.822258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.822233    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-certs\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.822524 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.822504    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.824160 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.824106    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-envoy\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.824661 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.824610    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-podinfo\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.830554 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.830526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c27gh\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-kube-api-access-c27gh\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.835014 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.834983    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-token\") pod \"e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") " pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:43.907757 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:43.907715    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:44.191370 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:44.191341    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"]
Sep 08 20:52:44.192520 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:52:44.192492    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ded1b8_b013_4f4a_9bee_4400c4e0e6e0.slice/crio-a5b2e8369c5d028df200e7763cd0b8d1d610360dfe8d837fdd98f7e46def4f93 WatchSource:0}: Error finding container a5b2e8369c5d028df200e7763cd0b8d1d610360dfe8d837fdd98f7e46def4f93: Status 404 returned error can't find the container with id a5b2e8369c5d028df200e7763cd0b8d1d610360dfe8d837fdd98f7e46def4f93
Sep 08 20:52:44.195022 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:44.194987    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:52:44.195137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:44.195075    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:52:44.195137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:44.195122    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:52:44.465064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:44.464936    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" event={"ID":"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0","Type":"ContainerStarted","Data":"17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9"}
Sep 08 20:52:44.465064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:44.465054    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" event={"ID":"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0","Type":"ContainerStarted","Data":"a5b2e8369c5d028df200e7763cd0b8d1d610360dfe8d837fdd98f7e46def4f93"}
Sep 08 20:52:44.489002 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:44.488935    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" podStartSLOduration=1.488912737 podStartE2EDuration="1.488912737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:52:44.484697989 +0000 UTC m=+1650.649868591" watchObservedRunningTime="2026-09-08 20:52:44.488912737 +0000 UTC m=+1650.654083307"
Sep 08 20:52:44.908757 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:44.908711    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:45.914201 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:45.914166    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:46.479237 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:46.479197    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:46.480118 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:46.480096    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:52:48.317662 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.317623    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"]
Sep 08 20:52:48.321452 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.321431    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:48.324204 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.324168    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4126a9-dockercfg-wd6ds\""
Sep 08 20:52:48.324375 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.324321    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4126a9\""
Sep 08 20:52:48.339363 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.339323    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"]
Sep 08 20:52:48.472729 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.472680    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk\" (UID: \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:48.472951 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.472758    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbcxk\" (UniqueName: \"kubernetes.io/projected/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-kube-api-access-gbcxk\") pod \"payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk\" (UID: \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:48.574162 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.574070    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk\" (UID: \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:48.574162 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.574140    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbcxk\" (UniqueName: \"kubernetes.io/projected/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-kube-api-access-gbcxk\") pod \"payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk\" (UID: \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:48.574980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.574957    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk\" (UID: \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:48.583176 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.583146    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbcxk\" (UniqueName: \"kubernetes.io/projected/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-kube-api-access-gbcxk\") pod \"payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk\" (UID: \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:48.635002 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.634958    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:48.926124 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:48.923958    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"]
Sep 08 20:52:49.500548 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:49.500503    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk" event={"ID":"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c","Type":"ContainerStarted","Data":"d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e"}
Sep 08 20:52:49.501036 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:49.500571    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:52:49.501036 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:49.500595    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk" event={"ID":"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c","Type":"ContainerStarted","Data":"a94afd5d1f6d8272817d6dff903d764ae3198620f4ad06089f655191aa2cba0f"}
Sep 08 20:52:49.520969 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:52:49.520912    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk" podStartSLOduration=1.520876176 podStartE2EDuration="1.520876176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:52:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:52:49.516781556 +0000 UTC m=+1655.681952143" watchObservedRunningTime="2026-09-08 20:52:49.520876176 +0000 UTC m=+1655.686046751"
Sep 08 20:53:00.509596 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:00.509559    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:53:07.632279 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:07.632246    2561 generic.go:356] "Generic (PLEG): container finished" podID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerID="17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9" exitCode=0
Sep 08 20:53:07.632713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:07.632323    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" event={"ID":"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0","Type":"ContainerDied","Data":"17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9"}
Sep 08 20:53:07.632834 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:07.632818    2561 scope.go:117] "RemoveContainer" containerID="17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9"
Sep 08 20:53:07.634052 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:07.634026    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:53:07.634126 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:07.634097    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:53:07.634173 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:07.634130    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 20:53:08.648132 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:08.648088    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" event={"ID":"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0","Type":"ContainerStarted","Data":"0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a"}
Sep 08 20:53:08.908388 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:08.908354    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:53:08.908595 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:08.908569    2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Sep 08 20:53:08.908680 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:08.908632    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Sep 08 20:53:09.908369 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:09.908330    2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Sep 08 20:53:09.908795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:09.908394    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Sep 08 20:53:10.908290 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:10.908249    2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Sep 08 20:53:10.908493 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:10.908307    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Sep 08 20:53:11.908138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:11.908098    2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Sep 08 20:53:11.908361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:11.908181    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Sep 08 20:53:12.004739 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.004699    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"]
Sep 08 20:53:12.009763 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.009734    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"
Sep 08 20:53:12.012360 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.012332    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:53:12.014141 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.014094    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"]
Sep 08 20:53:12.114976 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.114936    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ac3fd-9cf6-4bab-9ca8-84d72a62297e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855\" (UID: \"c54ac3fd-9cf6-4bab-9ca8-84d72a62297e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"
Sep 08 20:53:12.215859 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.215773    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ac3fd-9cf6-4bab-9ca8-84d72a62297e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855\" (UID: \"c54ac3fd-9cf6-4bab-9ca8-84d72a62297e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"
Sep 08 20:53:12.216439 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.216418    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ac3fd-9cf6-4bab-9ca8-84d72a62297e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855\" (UID: \"c54ac3fd-9cf6-4bab-9ca8-84d72a62297e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"
Sep 08 20:53:12.323811 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.323777    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"
Sep 08 20:53:12.622343 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.622311    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"]
Sep 08 20:53:12.623280 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:53:12.623255    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54ac3fd_9cf6_4bab_9ca8_84d72a62297e.slice/crio-d52a6f6f39d4e3f50584c34eae2ceb210fe785c223cf9516a5b2d705c059bf18 WatchSource:0}: Error finding container d52a6f6f39d4e3f50584c34eae2ceb210fe785c223cf9516a5b2d705c059bf18: Status 404 returned error can't find the container with id d52a6f6f39d4e3f50584c34eae2ceb210fe785c223cf9516a5b2d705c059bf18
Sep 08 20:53:12.686428 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.686396    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855" event={"ID":"c54ac3fd-9cf6-4bab-9ca8-84d72a62297e","Type":"ContainerStarted","Data":"d52a6f6f39d4e3f50584c34eae2ceb210fe785c223cf9516a5b2d705c059bf18"}
Sep 08 20:53:12.908484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.908448    2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Sep 08 20:53:12.908698 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:12.908503    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Sep 08 20:53:13.696017 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:13.695978    2561 generic.go:356] "Generic (PLEG): container finished" podID="c54ac3fd-9cf6-4bab-9ca8-84d72a62297e" containerID="9313b104d516556df3adfd9b1cb92b38fe4b86adc4b3c0c7a525ccc8c2ff13f4" exitCode=0
Sep 08 20:53:13.696429 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:13.696063    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855" event={"ID":"c54ac3fd-9cf6-4bab-9ca8-84d72a62297e","Type":"ContainerDied","Data":"9313b104d516556df3adfd9b1cb92b38fe4b86adc4b3c0c7a525ccc8c2ff13f4"}
Sep 08 20:53:13.912321 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:13.912291    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:53:13.912763 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:13.912745    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:53:13.913330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:13.913314    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:53:14.853146 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:14.853117    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"
Sep 08 20:53:14.943369 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:14.943333    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ac3fd-9cf6-4bab-9ca8-84d72a62297e-maas-api-service-ca\") pod \"c54ac3fd-9cf6-4bab-9ca8-84d72a62297e\" (UID: \"c54ac3fd-9cf6-4bab-9ca8-84d72a62297e\") "
Sep 08 20:53:14.943702 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:14.943678    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54ac3fd-9cf6-4bab-9ca8-84d72a62297e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c54ac3fd-9cf6-4bab-9ca8-84d72a62297e" (UID: "c54ac3fd-9cf6-4bab-9ca8-84d72a62297e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:53:15.044955 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:15.044839    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ac3fd-9cf6-4bab-9ca8-84d72a62297e-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:15.710299 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:15.710267    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"
Sep 08 20:53:15.710482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:15.710306    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855" event={"ID":"c54ac3fd-9cf6-4bab-9ca8-84d72a62297e","Type":"ContainerDied","Data":"d52a6f6f39d4e3f50584c34eae2ceb210fe785c223cf9516a5b2d705c059bf18"}
Sep 08 20:53:15.710482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:15.710353    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d52a6f6f39d4e3f50584c34eae2ceb210fe785c223cf9516a5b2d705c059bf18"
Sep 08 20:53:18.241100 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.241055    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"]
Sep 08 20:53:18.241650 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.241351    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk" podUID="8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c" containerName="payload-processing" containerID="cri-o://d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e" gracePeriod=30
Sep 08 20:53:18.412282 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.412238    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"]
Sep 08 20:53:18.416330 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.416166    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4126a9-fg855"]
Sep 08 20:53:18.527968 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.527940    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:53:18.685832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.685791    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbcxk\" (UniqueName: \"kubernetes.io/projected/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-kube-api-access-gbcxk\") pod \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\" (UID: \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\") "
Sep 08 20:53:18.686083 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.685875    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-plugins-config-volume\") pod \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\" (UID: \"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c\") "
Sep 08 20:53:18.686325 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.686298    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c" (UID: "8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:53:18.688110 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.688081    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-kube-api-access-gbcxk" (OuterVolumeSpecName: "kube-api-access-gbcxk") pod "8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c" (UID: "8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c"). InnerVolumeSpecName "kube-api-access-gbcxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:53:18.732332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.732296    2561 generic.go:356] "Generic (PLEG): container finished" podID="8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c" containerID="d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e" exitCode=0
Sep 08 20:53:18.732523 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.732385    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"
Sep 08 20:53:18.732523 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.732393    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk" event={"ID":"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c","Type":"ContainerDied","Data":"d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e"}
Sep 08 20:53:18.732523 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.732463    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk" event={"ID":"8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c","Type":"ContainerDied","Data":"a94afd5d1f6d8272817d6dff903d764ae3198620f4ad06089f655191aa2cba0f"}
Sep 08 20:53:18.732523 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.732497    2561 scope.go:117] "RemoveContainer" containerID="d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e"
Sep 08 20:53:18.743290 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.743243    2561 scope.go:117] "RemoveContainer" containerID="d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e"
Sep 08 20:53:18.743604 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:53:18.743580    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e\": container with ID starting with d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e not found: ID does not exist" containerID="d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e"
Sep 08 20:53:18.743656 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.743619    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e"} err="failed to get container status \"d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e\": rpc error: code = NotFound desc = could not find container \"d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e\": container with ID starting with d16d1e59169247bfc6efa6571add78e7d79c942abdaa794bc7084a124de25b0e not found: ID does not exist"
Sep 08 20:53:18.757135 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.757099    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"]
Sep 08 20:53:18.763672 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.763634    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4126a9-d7bc45bb9-57htk"]
Sep 08 20:53:18.787632 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.787594    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbcxk\" (UniqueName: \"kubernetes.io/projected/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-kube-api-access-gbcxk\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:18.787798 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:18.787635    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:20.364283 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:20.364245    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c" path="/var/lib/kubelet/pods/8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c/volumes"
Sep 08 20:53:20.364700 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:20.364597    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54ac3fd-9cf6-4bab-9ca8-84d72a62297e" path="/var/lib/kubelet/pods/c54ac3fd-9cf6-4bab-9ca8-84d72a62297e/volumes"
Sep 08 20:53:25.350748 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:25.350718    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"]
Sep 08 20:53:25.351160 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:25.350998    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy" containerID="cri-o://0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a" gracePeriod=30
Sep 08 20:53:28.914885 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:28.914846    2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 20:53:28.915287 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:28.914930    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 20:53:29.154964 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.154924    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"]
Sep 08 20:53:29.155538 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.155517    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54ac3fd-9cf6-4bab-9ca8-84d72a62297e" containerName="revoke-keys"
Sep 08 20:53:29.155638 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.155541    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54ac3fd-9cf6-4bab-9ca8-84d72a62297e" containerName="revoke-keys"
Sep 08 20:53:29.155638 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.155554    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c" containerName="payload-processing"
Sep 08 20:53:29.155638 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.155562    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c" containerName="payload-processing"
Sep 08 20:53:29.155790 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.155694    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54ac3fd-9cf6-4bab-9ca8-84d72a62297e" containerName="revoke-keys"
Sep 08 20:53:29.155790 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.155709    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b97cfc1-7cdf-4fc1-9b19-fc89233bd51c" containerName="payload-processing"
Sep 08 20:53:29.160220 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.160199    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"
Sep 08 20:53:29.164840 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.164814    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:53:29.168805 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.168724    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"]
Sep 08 20:53:29.294826 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.294781    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00bea595-77c3-4675-b532-64f53f7abd0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aab326-757jt\" (UID: \"00bea595-77c3-4675-b532-64f53f7abd0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"
Sep 08 20:53:29.395490 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.395449    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00bea595-77c3-4675-b532-64f53f7abd0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aab326-757jt\" (UID: \"00bea595-77c3-4675-b532-64f53f7abd0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"
Sep 08 20:53:29.396185 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.396156    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00bea595-77c3-4675-b532-64f53f7abd0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aab326-757jt\" (UID: \"00bea595-77c3-4675-b532-64f53f7abd0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"
Sep 08 20:53:29.474321 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.474238    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"
Sep 08 20:53:29.758419 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.757073    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"]
Sep 08 20:53:29.815581 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:29.815548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt" event={"ID":"00bea595-77c3-4675-b532-64f53f7abd0c","Type":"ContainerStarted","Data":"eaabb9186c06369a73dfac9597371d55c0d5bd6d7fa0852d66d1d0790f9c7fc4"}
Sep 08 20:53:30.614944 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.614867    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:53:30.811052 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811018    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istiod-ca-cert\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811248 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811060    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-credential-socket\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811248 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811092    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-podinfo\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811248 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811108    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-socket\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811248 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811233    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-certs\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811488 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811285    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-token\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811488 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811349    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c27gh\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-kube-api-access-c27gh\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811488 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811370    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:53:30.811488 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811436    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-data\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811488 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811436    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:53:30.811488 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811477    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-envoy\") pod \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\" (UID: \"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0\") "
Sep 08 20:53:30.811806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811502    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:53:30.811806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811513    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:53:30.811806 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811711    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:53:30.812006 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811953    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:30.812006 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811972    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:30.812006 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.811987    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:30.812006 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.812000    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:30.812224 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.812013    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:30.813569 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.813529    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:53:30.813569 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.813548    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-kube-api-access-c27gh" (OuterVolumeSpecName: "kube-api-access-c27gh") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "kube-api-access-c27gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:53:30.813739 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.813570    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:53:30.813739 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.813716    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" (UID: "37ded1b8-b013-4f4a-9bee-4400c4e0e6e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:53:30.826137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.826104    2561 generic.go:356] "Generic (PLEG): container finished" podID="00bea595-77c3-4675-b532-64f53f7abd0c" containerID="918e59dafdb17b48d30e8ddf1f37540c560343d215575d0a1443437f9a42f52d" exitCode=0
Sep 08 20:53:30.826254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.826181    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt" event={"ID":"00bea595-77c3-4675-b532-64f53f7abd0c","Type":"ContainerDied","Data":"918e59dafdb17b48d30e8ddf1f37540c560343d215575d0a1443437f9a42f52d"}
Sep 08 20:53:30.828182 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.828157    2561 generic.go:356] "Generic (PLEG): container finished" podID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerID="0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a" exitCode=0
Sep 08 20:53:30.828312 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.828205    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"
Sep 08 20:53:30.828312 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.828239    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" event={"ID":"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0","Type":"ContainerDied","Data":"0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a"}
Sep 08 20:53:30.828312 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.828290    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk" event={"ID":"37ded1b8-b013-4f4a-9bee-4400c4e0e6e0","Type":"ContainerDied","Data":"a5b2e8369c5d028df200e7763cd0b8d1d610360dfe8d837fdd98f7e46def4f93"}
Sep 08 20:53:30.828312 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.828311    2561 scope.go:117] "RemoveContainer" containerID="0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a"
Sep 08 20:53:30.840361 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.840342    2561 scope.go:117] "RemoveContainer" containerID="17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9"
Sep 08 20:53:30.850228 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.850214    2561 scope.go:117] "RemoveContainer" containerID="0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a"
Sep 08 20:53:30.850519 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:53:30.850500    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a\": container with ID starting with 0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a not found: ID does not exist" containerID="0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a"
Sep 08 20:53:30.850572 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.850525    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a"} err="failed to get container status \"0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a\": rpc error: code = NotFound desc = could not find container \"0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a\": container with ID starting with 0cede4e0a95686838300a2972c06306da25179dc888b7d5b4f651abee0936f5a not found: ID does not exist"
Sep 08 20:53:30.850572 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.850541    2561 scope.go:117] "RemoveContainer" containerID="17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9"
Sep 08 20:53:30.850772 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:53:30.850754    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9\": container with ID starting with 17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9 not found: ID does not exist" containerID="17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9"
Sep 08 20:53:30.850865 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.850781    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9"} err="failed to get container status \"17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9\": rpc error: code = NotFound desc = could not find container \"17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9\": container with ID starting with 17cb35a2e9ee4dbc8c08ef75240fc26029e72c1f695938449b59ae21d64a41d9 not found: ID does not exist"
Sep 08 20:53:30.861208 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.861187    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"]
Sep 08 20:53:30.866504 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.866451    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4126a9-openshift-default-d9b497b46-n9znk"]
Sep 08 20:53:30.913460 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.913426    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:30.913460 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.913456    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:30.913460 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.913466    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c27gh\" (UniqueName: \"kubernetes.io/projected/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-kube-api-access-c27gh\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:30.913682 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:30.913476    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:31.989860 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:31.989837    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"
Sep 08 20:53:32.021980 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:32.021953    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00bea595-77c3-4675-b532-64f53f7abd0c-maas-api-service-ca\") pod \"00bea595-77c3-4675-b532-64f53f7abd0c\" (UID: \"00bea595-77c3-4675-b532-64f53f7abd0c\") "
Sep 08 20:53:32.022350 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:32.022329    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00bea595-77c3-4675-b532-64f53f7abd0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "00bea595-77c3-4675-b532-64f53f7abd0c" (UID: "00bea595-77c3-4675-b532-64f53f7abd0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:53:32.123267 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:32.123173    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00bea595-77c3-4675-b532-64f53f7abd0c-maas-api-service-ca\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:32.364585 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:32.364549    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" path="/var/lib/kubelet/pods/37ded1b8-b013-4f4a-9bee-4400c4e0e6e0/volumes"
Sep 08 20:53:32.846108 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:32.846077    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"
Sep 08 20:53:32.846108 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:32.846102    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt" event={"ID":"00bea595-77c3-4675-b532-64f53f7abd0c","Type":"ContainerDied","Data":"eaabb9186c06369a73dfac9597371d55c0d5bd6d7fa0852d66d1d0790f9c7fc4"}
Sep 08 20:53:32.846318 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:32.846151    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eaabb9186c06369a73dfac9597371d55c0d5bd6d7fa0852d66d1d0790f9c7fc4"
Sep 08 20:53:44.768940 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:44.768876    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"]
Sep 08 20:53:44.769480 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:44.769184    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b" podUID="3cbe9fa6-6645-4f3a-85e9-838dff529487" containerName="payload-pre-processing" containerID="cri-o://877953ebdf5373304385e612beeea81d23610855193c7f2c0c055faccbcf1a70" gracePeriod=30
Sep 08 20:53:44.931766 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:44.931722    2561 generic.go:356] "Generic (PLEG): container finished" podID="3cbe9fa6-6645-4f3a-85e9-838dff529487" containerID="877953ebdf5373304385e612beeea81d23610855193c7f2c0c055faccbcf1a70" exitCode=0
Sep 08 20:53:44.931958 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:44.931795    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b" event={"ID":"3cbe9fa6-6645-4f3a-85e9-838dff529487","Type":"ContainerDied","Data":"877953ebdf5373304385e612beeea81d23610855193c7f2c0c055faccbcf1a70"}
Sep 08 20:53:44.968417 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:44.968387    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"]
Sep 08 20:53:44.971438 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:44.971412    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab326-757jt"]
Sep 08 20:53:45.052056 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.051938    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:53:45.146203 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.146167    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmk6k\" (UniqueName: \"kubernetes.io/projected/3cbe9fa6-6645-4f3a-85e9-838dff529487-kube-api-access-jmk6k\") pod \"3cbe9fa6-6645-4f3a-85e9-838dff529487\" (UID: \"3cbe9fa6-6645-4f3a-85e9-838dff529487\") "
Sep 08 20:53:45.146379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.146250    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cbe9fa6-6645-4f3a-85e9-838dff529487-plugins-config-volume\") pod \"3cbe9fa6-6645-4f3a-85e9-838dff529487\" (UID: \"3cbe9fa6-6645-4f3a-85e9-838dff529487\") "
Sep 08 20:53:45.146614 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.146591    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cbe9fa6-6645-4f3a-85e9-838dff529487-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3cbe9fa6-6645-4f3a-85e9-838dff529487" (UID: "3cbe9fa6-6645-4f3a-85e9-838dff529487"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:53:45.148355 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.148336    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cbe9fa6-6645-4f3a-85e9-838dff529487-kube-api-access-jmk6k" (OuterVolumeSpecName: "kube-api-access-jmk6k") pod "3cbe9fa6-6645-4f3a-85e9-838dff529487" (UID: "3cbe9fa6-6645-4f3a-85e9-838dff529487"). InnerVolumeSpecName "kube-api-access-jmk6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:53:45.247487 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.247449    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmk6k\" (UniqueName: \"kubernetes.io/projected/3cbe9fa6-6645-4f3a-85e9-838dff529487-kube-api-access-jmk6k\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:45.247487 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.247483    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cbe9fa6-6645-4f3a-85e9-838dff529487-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:53:45.949545 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.949514    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"
Sep 08 20:53:45.949545 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.949530    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b" event={"ID":"3cbe9fa6-6645-4f3a-85e9-838dff529487","Type":"ContainerDied","Data":"0cf692454a49eb0aa2760f95b62b967ccecfded76c80ec958e7c647b9da7f61c"}
Sep 08 20:53:45.950064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.949622    2561 scope.go:117] "RemoveContainer" containerID="877953ebdf5373304385e612beeea81d23610855193c7f2c0c055faccbcf1a70"
Sep 08 20:53:45.980739 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.980709    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"]
Sep 08 20:53:45.984779 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:45.984755    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab326-89bbd666b-jlt7b"]
Sep 08 20:53:46.363930 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:46.363833    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00bea595-77c3-4675-b532-64f53f7abd0c" path="/var/lib/kubelet/pods/00bea595-77c3-4675-b532-64f53f7abd0c/volumes"
Sep 08 20:53:46.364212 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:53:46.364195    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cbe9fa6-6645-4f3a-85e9-838dff529487" path="/var/lib/kubelet/pods/3cbe9fa6-6645-4f3a-85e9-838dff529487/volumes"
Sep 08 20:54:14.968337 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:14.968296    2561 scope.go:117] "RemoveContainer" containerID="091126a6bfa810092e165fa26166414cff558e449949093a8764311eea152a22"
Sep 08 20:54:34.072483 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:34.072449    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"]
Sep 08 20:54:34.072961 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:34.072700    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy" containerID="cri-o://d506755062f447b362ac5a6f8986ac765b7f8e0f11d766d0fe9a910c605f4e59" gracePeriod=30
Sep 08 20:54:39.315040 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.315009    2561 generic.go:356] "Generic (PLEG): container finished" podID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerID="d506755062f447b362ac5a6f8986ac765b7f8e0f11d766d0fe9a910c605f4e59" exitCode=0
Sep 08 20:54:39.315387 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.315081    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" event={"ID":"4009667b-5916-45cc-b2f6-ffe08e05b731","Type":"ContainerDied","Data":"d506755062f447b362ac5a6f8986ac765b7f8e0f11d766d0fe9a910c605f4e59"}
Sep 08 20:54:39.315387 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.315145    2561 scope.go:117] "RemoveContainer" containerID="d683dba9995fba1ac195d4c960ddcf4a84468223ea4ed389220213a6572b0d82"
Sep 08 20:54:39.339465 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.339441    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:54:39.488367 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488335    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-podinfo\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488410    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-credential-socket\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488430    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-socket\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488447    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-data\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488487    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsrjg\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-kube-api-access-tsrjg\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488567 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488552    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-envoy\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488836 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488571    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-token\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488836 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488614    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4009667b-5916-45cc-b2f6-ffe08e05b731-istiod-ca-cert\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488836 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488640    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-certs\") pod \"4009667b-5916-45cc-b2f6-ffe08e05b731\" (UID: \"4009667b-5916-45cc-b2f6-ffe08e05b731\") "
Sep 08 20:54:39.488836 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488760    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-data" (OuterVolumeSpecName: "istio-data") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:39.488836 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.488774    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:39.489157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.489017    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:39.489157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.489068    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:39.489157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.489132    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4009667b-5916-45cc-b2f6-ffe08e05b731-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:54:39.489329 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.489158    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:39.489329 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.489179    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:39.489329 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.489205    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:39.490769 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.490740    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:54:39.490881 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.490767    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-kube-api-access-tsrjg" (OuterVolumeSpecName: "kube-api-access-tsrjg") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "kube-api-access-tsrjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:54:39.490881 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.490783    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-token" (OuterVolumeSpecName: "istio-token") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:54:39.490881 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.490847    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4009667b-5916-45cc-b2f6-ffe08e05b731" (UID: "4009667b-5916-45cc-b2f6-ffe08e05b731"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:39.589781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.589698    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:39.589781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.589724    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:39.589781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.589735    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsrjg\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-kube-api-access-tsrjg\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:39.589781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.589744    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:39.589781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.589753    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4009667b-5916-45cc-b2f6-ffe08e05b731-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:39.589781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:39.589762    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4009667b-5916-45cc-b2f6-ffe08e05b731-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:40.323645 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:40.323604    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"
Sep 08 20:54:40.324206 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:40.323651    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv" event={"ID":"4009667b-5916-45cc-b2f6-ffe08e05b731","Type":"ContainerDied","Data":"0307386ec1de204a617aa99663f86a853749a5e1dfd25eaddc22570677909eca"}
Sep 08 20:54:40.324206 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:40.323701    2561 scope.go:117] "RemoveContainer" containerID="d506755062f447b362ac5a6f8986ac765b7f8e0f11d766d0fe9a910c605f4e59"
Sep 08 20:54:40.371484 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:40.370774    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"]
Sep 08 20:54:40.374783 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:40.374745    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab326-openshift-default-766b8794b8-mmrtv"]
Sep 08 20:54:40.382014 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:54:40.381983    2561 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4009667b_5916_45cc_b2f6_ffe08e05b731.slice/crio-0307386ec1de204a617aa99663f86a853749a5e1dfd25eaddc22570677909eca\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4009667b_5916_45cc_b2f6_ffe08e05b731.slice\": RecentStats: unable to find data in memory cache]"
Sep 08 20:54:42.364206 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:42.364166    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" path="/var/lib/kubelet/pods/4009667b-5916-45cc-b2f6-ffe08e05b731/volumes"
Sep 08 20:54:45.751706 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:45.751670    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"]
Sep 08 20:54:45.752173 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:45.751930    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx" podUID="c2533658-613b-4b9f-817e-5306e3525b44" containerName="payload-processing" containerID="cri-o://89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb" gracePeriod=30
Sep 08 20:54:45.773744 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:45.773719    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"]
Sep 08 20:54:45.773969 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:45.773945    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg" podUID="4eb4fa70-e792-42f4-9a6d-c176c5b2ea17" containerName="payload-pre-processing" containerID="cri-o://d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18" gracePeriod=30
Sep 08 20:54:46.071220 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.071192    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:54:46.074256 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.074235    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:54:46.256639 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.256605    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-plugins-config-volume\") pod \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\" (UID: \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\") "
Sep 08 20:54:46.256865 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.256670    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2533658-613b-4b9f-817e-5306e3525b44-plugins-config-volume\") pod \"c2533658-613b-4b9f-817e-5306e3525b44\" (UID: \"c2533658-613b-4b9f-817e-5306e3525b44\") "
Sep 08 20:54:46.256865 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.256778    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrsr4\" (UniqueName: \"kubernetes.io/projected/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-kube-api-access-zrsr4\") pod \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\" (UID: \"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17\") "
Sep 08 20:54:46.257057 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.256927    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfzpn\" (UniqueName: \"kubernetes.io/projected/c2533658-613b-4b9f-817e-5306e3525b44-kube-api-access-wfzpn\") pod \"c2533658-613b-4b9f-817e-5306e3525b44\" (UID: \"c2533658-613b-4b9f-817e-5306e3525b44\") "
Sep 08 20:54:46.257138 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.257114    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4eb4fa70-e792-42f4-9a6d-c176c5b2ea17" (UID: "4eb4fa70-e792-42f4-9a6d-c176c5b2ea17"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:54:46.257201 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.257173    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2533658-613b-4b9f-817e-5306e3525b44-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c2533658-613b-4b9f-817e-5306e3525b44" (UID: "c2533658-613b-4b9f-817e-5306e3525b44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:54:46.257365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.257348    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:46.257436 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.257371    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2533658-613b-4b9f-817e-5306e3525b44-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:46.259051 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.259029    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-kube-api-access-zrsr4" (OuterVolumeSpecName: "kube-api-access-zrsr4") pod "4eb4fa70-e792-42f4-9a6d-c176c5b2ea17" (UID: "4eb4fa70-e792-42f4-9a6d-c176c5b2ea17"). InnerVolumeSpecName "kube-api-access-zrsr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:54:46.259141 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.259059    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2533658-613b-4b9f-817e-5306e3525b44-kube-api-access-wfzpn" (OuterVolumeSpecName: "kube-api-access-wfzpn") pod "c2533658-613b-4b9f-817e-5306e3525b44" (UID: "c2533658-613b-4b9f-817e-5306e3525b44"). InnerVolumeSpecName "kube-api-access-wfzpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:54:46.358103 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.358015    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrsr4\" (UniqueName: \"kubernetes.io/projected/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17-kube-api-access-zrsr4\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:46.358103 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.358054    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfzpn\" (UniqueName: \"kubernetes.io/projected/c2533658-613b-4b9f-817e-5306e3525b44-kube-api-access-wfzpn\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:46.373435 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.373388    2561 generic.go:356] "Generic (PLEG): container finished" podID="c2533658-613b-4b9f-817e-5306e3525b44" containerID="89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb" exitCode=0
Sep 08 20:54:46.373435 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.373445    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:54:46.373679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.373477    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx" event={"ID":"c2533658-613b-4b9f-817e-5306e3525b44","Type":"ContainerDied","Data":"89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb"}
Sep 08 20:54:46.373679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.373539    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx" event={"ID":"c2533658-613b-4b9f-817e-5306e3525b44","Type":"ContainerDied","Data":"6b6a976a776f5290c5317ea0bb217057da390e57b16c7488a9565788333c9b83"}
Sep 08 20:54:46.373679 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.373569    2561 scope.go:117] "RemoveContainer" containerID="89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb"
Sep 08 20:54:46.379935 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.379910    2561 generic.go:356] "Generic (PLEG): container finished" podID="4eb4fa70-e792-42f4-9a6d-c176c5b2ea17" containerID="d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18" exitCode=0
Sep 08 20:54:46.380064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.380025    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg" event={"ID":"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17","Type":"ContainerDied","Data":"d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18"}
Sep 08 20:54:46.380064 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.380030    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"
Sep 08 20:54:46.380176 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.380064    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg" event={"ID":"4eb4fa70-e792-42f4-9a6d-c176c5b2ea17","Type":"ContainerDied","Data":"279fc070df86ccd8b430c07c9de834b2bf943a248f1111b40927e991042c0533"}
Sep 08 20:54:46.390296 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.390280    2561 scope.go:117] "RemoveContainer" containerID="89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb"
Sep 08 20:54:46.390560 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:54:46.390542    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb\": container with ID starting with 89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb not found: ID does not exist" containerID="89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb"
Sep 08 20:54:46.390636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.390573    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb"} err="failed to get container status \"89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb\": rpc error: code = NotFound desc = could not find container \"89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb\": container with ID starting with 89deee9d3b0119d5d33c84945d35b932c8acb511141040d90404349eb52da6fb not found: ID does not exist"
Sep 08 20:54:46.390636 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.390596    2561 scope.go:117] "RemoveContainer" containerID="d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18"
Sep 08 20:54:46.399386 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.399367    2561 scope.go:117] "RemoveContainer" containerID="d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18"
Sep 08 20:54:46.399664 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:54:46.399646    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18\": container with ID starting with d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18 not found: ID does not exist" containerID="d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18"
Sep 08 20:54:46.399713 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.399675    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18"} err="failed to get container status \"d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18\": rpc error: code = NotFound desc = could not find container \"d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18\": container with ID starting with d44f02e4c7e583a056989021571b5d763f9e4f384a42e208e0965ab5dfd3ff18 not found: ID does not exist"
Sep 08 20:54:46.420997 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.420968    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"]
Sep 08 20:54:46.427930 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:46.427909    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0566ff-79b49f68cd-fp5tg"]
Sep 08 20:54:48.363819 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:48.363786    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eb4fa70-e792-42f4-9a6d-c176c5b2ea17" path="/var/lib/kubelet/pods/4eb4fa70-e792-42f4-9a6d-c176c5b2ea17/volumes"
Sep 08 20:54:53.426828 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:53.426797    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"]
Sep 08 20:54:53.427315 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:53.427059    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy" containerID="cri-o://3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796" gracePeriod=30
Sep 08 20:54:58.694225 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.694151    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:54:58.778011 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.777978    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-data\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778011 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778012    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5f2435a-d50c-484a-a9ca-b426bf19009d-istiod-ca-cert\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778057    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-token\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778080    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-envoy\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778095    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-certs\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778133    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-credential-socket\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778178    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-podinfo\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778254 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778236    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtc2h\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-kube-api-access-rtc2h\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778285    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-socket\") pod \"f5f2435a-d50c-484a-a9ca-b426bf19009d\" (UID: \"f5f2435a-d50c-484a-a9ca-b426bf19009d\") "
Sep 08 20:54:58.778550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778436    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:58.778550 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778474    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:58.778698 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778655    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:58.778698 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778662    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:58.778698 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778692    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-data\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:58.778911 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778876    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:58.778988 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.778952    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5f2435a-d50c-484a-a9ca-b426bf19009d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:54:58.780621 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.780586    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:58.780621 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.780616    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:54:58.780780 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.780664    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-kube-api-access-rtc2h" (OuterVolumeSpecName: "kube-api-access-rtc2h") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "kube-api-access-rtc2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:54:58.780780 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.780677    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5f2435a-d50c-484a-a9ca-b426bf19009d" (UID: "f5f2435a-d50c-484a-a9ca-b426bf19009d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:54:58.880093 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.880050    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5f2435a-d50c-484a-a9ca-b426bf19009d-istiod-ca-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:58.880093 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.880084    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-token\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:58.880093 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.880093    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-envoy\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:58.880093 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.880103    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-credential-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:58.880378 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.880113    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5f2435a-d50c-484a-a9ca-b426bf19009d-istio-podinfo\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:58.880378 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.880122    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtc2h\" (UniqueName: \"kubernetes.io/projected/f5f2435a-d50c-484a-a9ca-b426bf19009d-kube-api-access-rtc2h\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:58.880378 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:58.880131    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5f2435a-d50c-484a-a9ca-b426bf19009d-workload-socket\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:54:59.470332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.470300    2561 generic.go:356] "Generic (PLEG): container finished" podID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerID="3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796" exitCode=0
Sep 08 20:54:59.470518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.470354    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"
Sep 08 20:54:59.470518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.470389    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" event={"ID":"f5f2435a-d50c-484a-a9ca-b426bf19009d","Type":"ContainerDied","Data":"3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796"}
Sep 08 20:54:59.470518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.470451    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p" event={"ID":"f5f2435a-d50c-484a-a9ca-b426bf19009d","Type":"ContainerDied","Data":"b682c00b4a4074b83e04eec396483305b8b57902f04ed2929d75e412118d061d"}
Sep 08 20:54:59.470518 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.470475    2561 scope.go:117] "RemoveContainer" containerID="3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796"
Sep 08 20:54:59.480649 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.480632    2561 scope.go:117] "RemoveContainer" containerID="72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c"
Sep 08 20:54:59.489674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.489657    2561 scope.go:117] "RemoveContainer" containerID="3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796"
Sep 08 20:54:59.489958 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:54:59.489938    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796\": container with ID starting with 3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796 not found: ID does not exist" containerID="3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796"
Sep 08 20:54:59.490015 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.489968    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796"} err="failed to get container status \"3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796\": rpc error: code = NotFound desc = could not find container \"3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796\": container with ID starting with 3993378015d10a35dcadbd941fc07188333c1e2faac325a8096dd00e49e69796 not found: ID does not exist"
Sep 08 20:54:59.490015 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.489987    2561 scope.go:117] "RemoveContainer" containerID="72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c"
Sep 08 20:54:59.490272 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:54:59.490251    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c\": container with ID starting with 72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c not found: ID does not exist" containerID="72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c"
Sep 08 20:54:59.490347 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.490277    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c"} err="failed to get container status \"72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c\": rpc error: code = NotFound desc = could not find container \"72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c\": container with ID starting with 72a1e4a50d397a8895f0e370be9481052fa815984c3ae2611d5f23972765ca3c not found: ID does not exist"
Sep 08 20:54:59.496530 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.496508    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"]
Sep 08 20:54:59.499324 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:54:59.499301    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-0566ff-openshift-default-64c7c6dd9d-9dq2p"]
Sep 08 20:55:00.365069 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:00.365036    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" path="/var/lib/kubelet/pods/f5f2435a-d50c-484a-a9ca-b426bf19009d/volumes"
Sep 08 20:55:04.440287 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.440238    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"]
Sep 08 20:55:04.440781 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.440492    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2" podUID="66ca5239-bcc9-4409-9eb5-308d0fa61cb3" containerName="payload-pre-processing" containerID="cri-o://3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27" gracePeriod=30
Sep 08 20:55:04.714062 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.714030    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:55:04.833151 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.833109    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-plugins-config-volume\") pod \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\" (UID: \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\") "
Sep 08 20:55:04.833368 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.833175    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bmk6\" (UniqueName: \"kubernetes.io/projected/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-kube-api-access-7bmk6\") pod \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\" (UID: \"66ca5239-bcc9-4409-9eb5-308d0fa61cb3\") "
Sep 08 20:55:04.833582 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.833554    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66ca5239-bcc9-4409-9eb5-308d0fa61cb3" (UID: "66ca5239-bcc9-4409-9eb5-308d0fa61cb3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:55:04.835390 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.835362    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-kube-api-access-7bmk6" (OuterVolumeSpecName: "kube-api-access-7bmk6") pod "66ca5239-bcc9-4409-9eb5-308d0fa61cb3" (UID: "66ca5239-bcc9-4409-9eb5-308d0fa61cb3"). InnerVolumeSpecName "kube-api-access-7bmk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:04.933937 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.933881    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:55:04.933937 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:04.933927    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bmk6\" (UniqueName: \"kubernetes.io/projected/66ca5239-bcc9-4409-9eb5-308d0fa61cb3-kube-api-access-7bmk6\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:55:05.513024 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.512986    2561 generic.go:356] "Generic (PLEG): container finished" podID="66ca5239-bcc9-4409-9eb5-308d0fa61cb3" containerID="3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27" exitCode=0
Sep 08 20:55:05.513492 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.513041    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"
Sep 08 20:55:05.513492 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.513046    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2" event={"ID":"66ca5239-bcc9-4409-9eb5-308d0fa61cb3","Type":"ContainerDied","Data":"3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27"}
Sep 08 20:55:05.513492 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.513102    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2" event={"ID":"66ca5239-bcc9-4409-9eb5-308d0fa61cb3","Type":"ContainerDied","Data":"23581f26e153b01e98682478e2bfa21fc4bde0b81452ab491993b93bc8b0dab2"}
Sep 08 20:55:05.513492 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.513132    2561 scope.go:117] "RemoveContainer" containerID="3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27"
Sep 08 20:55:05.524420 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.524401    2561 scope.go:117] "RemoveContainer" containerID="3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27"
Sep 08 20:55:05.524759 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:55:05.524741    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27\": container with ID starting with 3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27 not found: ID does not exist" containerID="3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27"
Sep 08 20:55:05.524821 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.524771    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27"} err="failed to get container status \"3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27\": rpc error: code = NotFound desc = could not find container \"3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27\": container with ID starting with 3b44ba32e97b9d6beb8019ec2836de01727149f0bdb24de7ec17925a601eab27 not found: ID does not exist"
Sep 08 20:55:05.537875 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.537841    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"]
Sep 08 20:55:05.543157 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:05.543119    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-78dd9f-6765798d7d-jkws2"]
Sep 08 20:55:06.364682 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:06.364603    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66ca5239-bcc9-4409-9eb5-308d0fa61cb3" path="/var/lib/kubelet/pods/66ca5239-bcc9-4409-9eb5-308d0fa61cb3/volumes"
Sep 08 20:55:14.550614 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:14.550584    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:55:14.556051 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:14.556020    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 20:55:16.386383 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:16.386335    2561 pod_container_manager_linux.go:217] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","podc2533658-613b-4b9f-817e-5306e3525b44"] err="unable to destroy cgroup paths for cgroup [kubepods burstable podc2533658-613b-4b9f-817e-5306e3525b44] : Timed out while waiting for systemd to remove kubepods-burstable-podc2533658_613b_4b9f_817e_5306e3525b44.slice"
Sep 08 20:55:16.386383 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:55:16.386384    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods burstable podc2533658-613b-4b9f-817e-5306e3525b44] : unable to destroy cgroup paths for cgroup [kubepods burstable podc2533658-613b-4b9f-817e-5306e3525b44] : Timed out while waiting for systemd to remove kubepods-burstable-podc2533658_613b_4b9f_817e_5306e3525b44.slice" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx" podUID="c2533658-613b-4b9f-817e-5306e3525b44"
Sep 08 20:55:16.593947 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:16.593917    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"
Sep 08 20:55:16.646652 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:16.646560    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"]
Sep 08 20:55:16.649123 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:16.649096    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0566ff-7c7f9bbf89-nqzsx"]
Sep 08 20:55:18.364200 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:18.364164    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2533658-613b-4b9f-817e-5306e3525b44" path="/var/lib/kubelet/pods/c2533658-613b-4b9f-817e-5306e3525b44/volumes"
Sep 08 20:55:23.221674 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.221637    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"]
Sep 08 20:55:23.232297 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.232260    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c9590b-29815005-r2dx5"]
Sep 08 20:55:23.278873 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.278835    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"]
Sep 08 20:55:23.279158 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.279128    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h" podUID="55102c6e-2363-428b-8f42-2d47f1638e72" containerName="payload-processing" containerID="cri-o://a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e" gracePeriod=30
Sep 08 20:55:23.546404 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.546376    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:55:23.639967 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.639924    2561 generic.go:356] "Generic (PLEG): container finished" podID="55102c6e-2363-428b-8f42-2d47f1638e72" containerID="a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e" exitCode=0
Sep 08 20:55:23.640137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.640022    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"
Sep 08 20:55:23.640137 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.640048    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h" event={"ID":"55102c6e-2363-428b-8f42-2d47f1638e72","Type":"ContainerDied","Data":"a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e"}
Sep 08 20:55:23.640216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.640138    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h" event={"ID":"55102c6e-2363-428b-8f42-2d47f1638e72","Type":"ContainerDied","Data":"a7ed85307ec4241e947455f2c4726abb28d3af3f04aa17e24f8c6819f12f9637"}
Sep 08 20:55:23.640216 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.640167    2561 scope.go:117] "RemoveContainer" containerID="a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e"
Sep 08 20:55:23.650066 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.650043    2561 scope.go:117] "RemoveContainer" containerID="a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e"
Sep 08 20:55:23.650444 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:55:23.650420    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e\": container with ID starting with a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e not found: ID does not exist" containerID="a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e"
Sep 08 20:55:23.650522 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.650453    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e"} err="failed to get container status \"a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e\": rpc error: code = NotFound desc = could not find container \"a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e\": container with ID starting with a817ab90a34c43c2ff502326882312bdd2cc1ad4e2f04a0986fd3e47c677635e not found: ID does not exist"
Sep 08 20:55:23.716716 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.716677    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55102c6e-2363-428b-8f42-2d47f1638e72-plugins-config-volume\") pod \"55102c6e-2363-428b-8f42-2d47f1638e72\" (UID: \"55102c6e-2363-428b-8f42-2d47f1638e72\") "
Sep 08 20:55:23.716881 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.716776    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhrt8\" (UniqueName: \"kubernetes.io/projected/55102c6e-2363-428b-8f42-2d47f1638e72-kube-api-access-xhrt8\") pod \"55102c6e-2363-428b-8f42-2d47f1638e72\" (UID: \"55102c6e-2363-428b-8f42-2d47f1638e72\") "
Sep 08 20:55:23.717205 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.717183    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55102c6e-2363-428b-8f42-2d47f1638e72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55102c6e-2363-428b-8f42-2d47f1638e72" (UID: "55102c6e-2363-428b-8f42-2d47f1638e72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:55:23.719011 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.718983    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55102c6e-2363-428b-8f42-2d47f1638e72-kube-api-access-xhrt8" (OuterVolumeSpecName: "kube-api-access-xhrt8") pod "55102c6e-2363-428b-8f42-2d47f1638e72" (UID: "55102c6e-2363-428b-8f42-2d47f1638e72"). InnerVolumeSpecName "kube-api-access-xhrt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:23.817622 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.817532    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhrt8\" (UniqueName: \"kubernetes.io/projected/55102c6e-2363-428b-8f42-2d47f1638e72-kube-api-access-xhrt8\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:55:23.817622 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.817563    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55102c6e-2363-428b-8f42-2d47f1638e72-plugins-config-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:55:23.974186 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.974143    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"]
Sep 08 20:55:23.982460 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:23.982426    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-c9590b-54f458788f-brl4h"]
Sep 08 20:55:24.365055 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:24.365019    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55102c6e-2363-428b-8f42-2d47f1638e72" path="/var/lib/kubelet/pods/55102c6e-2363-428b-8f42-2d47f1638e72/volumes"
Sep 08 20:55:24.365504 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:24.365363    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9d10b58-7790-424e-b1e3-21ec85f21b0b" path="/var/lib/kubelet/pods/e9d10b58-7790-424e-b1e3-21ec85f21b0b/volumes"
Sep 08 20:55:32.952759 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:32.952720    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-dxwnd"]
Sep 08 20:55:32.953229 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:32.952953    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd" podUID="d004915c-96ee-4263-bd9f-65258d2b2cc1" containerName="manager" containerID="cri-o://835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f" gracePeriod=30
Sep 08 20:55:33.210929 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.210837    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:55:33.307222 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.307173    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cvg5\" (UniqueName: \"kubernetes.io/projected/d004915c-96ee-4263-bd9f-65258d2b2cc1-kube-api-access-7cvg5\") pod \"d004915c-96ee-4263-bd9f-65258d2b2cc1\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") "
Sep 08 20:55:33.307424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.307231    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-metrics-certs\") pod \"d004915c-96ee-4263-bd9f-65258d2b2cc1\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") "
Sep 08 20:55:33.307424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.307354    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-webhook-cert\") pod \"d004915c-96ee-4263-bd9f-65258d2b2cc1\" (UID: \"d004915c-96ee-4263-bd9f-65258d2b2cc1\") "
Sep 08 20:55:33.309445 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.309400    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d004915c-96ee-4263-bd9f-65258d2b2cc1" (UID: "d004915c-96ee-4263-bd9f-65258d2b2cc1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:55:33.309583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.309474    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d004915c-96ee-4263-bd9f-65258d2b2cc1-kube-api-access-7cvg5" (OuterVolumeSpecName: "kube-api-access-7cvg5") pod "d004915c-96ee-4263-bd9f-65258d2b2cc1" (UID: "d004915c-96ee-4263-bd9f-65258d2b2cc1"). InnerVolumeSpecName "kube-api-access-7cvg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:33.309583 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.309490    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d004915c-96ee-4263-bd9f-65258d2b2cc1" (UID: "d004915c-96ee-4263-bd9f-65258d2b2cc1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:55:33.408604 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.408562    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-webhook-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:55:33.408604 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.408600    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cvg5\" (UniqueName: \"kubernetes.io/projected/d004915c-96ee-4263-bd9f-65258d2b2cc1-kube-api-access-7cvg5\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:55:33.408604 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.408616    2561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d004915c-96ee-4263-bd9f-65258d2b2cc1-metrics-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:55:33.712712 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.712675    2561 generic.go:356] "Generic (PLEG): container finished" podID="d004915c-96ee-4263-bd9f-65258d2b2cc1" containerID="835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f" exitCode=0
Sep 08 20:55:33.712950 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.712745    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd"
Sep 08 20:55:33.712950 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.712755    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd" event={"ID":"d004915c-96ee-4263-bd9f-65258d2b2cc1","Type":"ContainerDied","Data":"835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f"}
Sep 08 20:55:33.712950 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.712811    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-dxwnd" event={"ID":"d004915c-96ee-4263-bd9f-65258d2b2cc1","Type":"ContainerDied","Data":"497ac8135f7ac86048c1ca23f075384420486b0dad0c41b43806bc5c144ad246"}
Sep 08 20:55:33.712950 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.712834    2561 scope.go:117] "RemoveContainer" containerID="835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f"
Sep 08 20:55:33.723177 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.723153    2561 scope.go:117] "RemoveContainer" containerID="835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f"
Sep 08 20:55:33.723486 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:55:33.723465    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f\": container with ID starting with 835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f not found: ID does not exist" containerID="835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f"
Sep 08 20:55:33.723551 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.723502    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f"} err="failed to get container status \"835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f\": rpc error: code = NotFound desc = could not find container \"835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f\": container with ID starting with 835efe48c7136ba71bc7bd0f1d4279f743c1ff872cabe0f1b0d60f4499936c1f not found: ID does not exist"
Sep 08 20:55:33.735394 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.735360    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-dxwnd"]
Sep 08 20:55:33.739472 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:33.739442    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-dxwnd"]
Sep 08 20:55:34.364345 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:55:34.364312    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d004915c-96ee-4263-bd9f-65258d2b2cc1" path="/var/lib/kubelet/pods/d004915c-96ee-4263-bd9f-65258d2b2cc1/volumes"
Sep 08 20:56:15.041549 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:15.041519    2561 scope.go:117] "RemoveContainer" containerID="331dade7430abdce3944cfd9488eaebdc78954d795ddb320fa0f033a6a5714b0"
Sep 08 20:56:15.050598 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:15.050576    2561 scope.go:117] "RemoveContainer" containerID="5229ba28ce6b8ec00cbef5d4cf8c813a151e91a8072bbed6692610de879af257"
Sep 08 20:56:15.058930 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:15.058884    2561 scope.go:117] "RemoveContainer" containerID="d3e1713cddeaa6002599efb97b4cc48d80298601faf743d75b36c40eb80cc8f2"
Sep 08 20:56:15.066931 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:15.066913    2561 scope.go:117] "RemoveContainer" containerID="2828974e2d7b8c96b2a367d637a0f8f4bf12bc4b81ff99e67d40ad17678714f9"
Sep 08 20:56:34.573329 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573287    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-8qknb"]
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573820    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573837    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573846    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573851    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573859    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573865    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573880    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55102c6e-2363-428b-8f42-2d47f1638e72" containerName="payload-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573886    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="55102c6e-2363-428b-8f42-2d47f1638e72" containerName="payload-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573908    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cbe9fa6-6645-4f3a-85e9-838dff529487" containerName="payload-pre-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573917    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cbe9fa6-6645-4f3a-85e9-838dff529487" containerName="payload-pre-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573925    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ca5239-bcc9-4409-9eb5-308d0fa61cb3" containerName="payload-pre-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573931    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ca5239-bcc9-4409-9eb5-308d0fa61cb3" containerName="payload-pre-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573938    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2533658-613b-4b9f-817e-5306e3525b44" containerName="payload-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573944    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2533658-613b-4b9f-817e-5306e3525b44" containerName="payload-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573955    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb4fa70-e792-42f4-9a6d-c176c5b2ea17" containerName="payload-pre-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573960    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb4fa70-e792-42f4-9a6d-c176c5b2ea17" containerName="payload-pre-processing"
Sep 08 20:56:34.573960 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573968    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d004915c-96ee-4263-bd9f-65258d2b2cc1" containerName="manager"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573974    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d004915c-96ee-4263-bd9f-65258d2b2cc1" containerName="manager"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573982    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00bea595-77c3-4675-b532-64f53f7abd0c" containerName="revoke-keys"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.573987    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="00bea595-77c3-4675-b532-64f53f7abd0c" containerName="revoke-keys"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574052    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ca5239-bcc9-4409-9eb5-308d0fa61cb3" containerName="payload-pre-processing"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574060    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574066    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574072    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d004915c-96ee-4263-bd9f-65258d2b2cc1" containerName="manager"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574077    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cbe9fa6-6645-4f3a-85e9-838dff529487" containerName="payload-pre-processing"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574082    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2533658-613b-4b9f-817e-5306e3525b44" containerName="payload-processing"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574090    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574097    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574102    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574108    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="00bea595-77c3-4675-b532-64f53f7abd0c" containerName="revoke-keys"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574114    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574120    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="55102c6e-2363-428b-8f42-2d47f1638e72" containerName="payload-processing"
Sep 08 20:56:34.574482 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.574125    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb4fa70-e792-42f4-9a6d-c176c5b2ea17" containerName="payload-pre-processing"
Sep 08 20:56:34.577214 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.577197    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.581226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.581195    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 20:56:34.581226 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.581219    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 20:56:34.581412 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.581227    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qwqtd\""
Sep 08 20:56:34.581412 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.581245    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 20:56:34.590219 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.590188    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-8qknb"]
Sep 08 20:56:34.678360 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.678313    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-webhook-cert\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.678547 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.678369    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-metrics-certs\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.678547 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.678474    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mdwn\" (UniqueName: \"kubernetes.io/projected/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-kube-api-access-9mdwn\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.779641 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.779604    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdwn\" (UniqueName: \"kubernetes.io/projected/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-kube-api-access-9mdwn\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.779801 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.779681    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-webhook-cert\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.779801 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.779708    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-metrics-certs\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.782290 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.782253    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-metrics-certs\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.782429 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.782298    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-webhook-cert\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.788044 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.788016    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mdwn\" (UniqueName: \"kubernetes.io/projected/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-kube-api-access-9mdwn\") pod \"maas-controller-7cc45c6c68-8qknb\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") " pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:34.888983 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:34.888866    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:35.165776 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:35.165735    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-8qknb"]
Sep 08 20:56:35.166558 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:56:35.166534    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0ae04e_8c27_4185_8bd9_0dfb7f2ea879.slice/crio-a22378e4d02f3d3eb9df1d791dca8672b8c03abedc6b94a625f66dd9d6f0e937 WatchSource:0}: Error finding container a22378e4d02f3d3eb9df1d791dca8672b8c03abedc6b94a625f66dd9d6f0e937: Status 404 returned error can't find the container with id a22378e4d02f3d3eb9df1d791dca8672b8c03abedc6b94a625f66dd9d6f0e937
Sep 08 20:56:35.168136 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:35.168112    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:56:36.111428 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:36.111391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-8qknb" event={"ID":"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879","Type":"ContainerStarted","Data":"a22378e4d02f3d3eb9df1d791dca8672b8c03abedc6b94a625f66dd9d6f0e937"}
Sep 08 20:56:37.118966 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:37.118927    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-8qknb" event={"ID":"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879","Type":"ContainerStarted","Data":"2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d"}
Sep 08 20:56:37.119353 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:37.119006    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:37.140455 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:37.140390    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cc45c6c68-8qknb" podStartSLOduration=2.017041836 podStartE2EDuration="3.140370009s" totalImagesPullingTime="1.123328173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:56:37.138207947 +0000 UTC m=+1883.303378522" watchObservedRunningTime="2026-09-08 20:56:37.140370009 +0000 UTC m=+1883.305540584"
Sep 08 20:56:48.135791 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:48.135753    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:56:59.770646 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:59.770603    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-8qknb"]
Sep 08 20:56:59.771560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:56:59.771500    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7cc45c6c68-8qknb" podUID="ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" containerName="manager" containerID="cri-o://2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d" gracePeriod=30
Sep 08 20:57:00.037776 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.037693    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:57:00.127814 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.127777    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-metrics-certs\") pod \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") "
Sep 08 20:57:00.128046 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.127832    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mdwn\" (UniqueName: \"kubernetes.io/projected/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-kube-api-access-9mdwn\") pod \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") "
Sep 08 20:57:00.128046 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.127909    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-webhook-cert\") pod \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\" (UID: \"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879\") "
Sep 08 20:57:00.130087 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.130053    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" (UID: "ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:57:00.130471 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.130433    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-kube-api-access-9mdwn" (OuterVolumeSpecName: "kube-api-access-9mdwn") pod "ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" (UID: "ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879"). InnerVolumeSpecName "kube-api-access-9mdwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:57:00.130579 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.130491    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" (UID: "ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:57:00.229537 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.229492    2561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-metrics-certs\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:57:00.229537 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.229531    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mdwn\" (UniqueName: \"kubernetes.io/projected/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-kube-api-access-9mdwn\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:57:00.229537 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.229541    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879-webhook-cert\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:57:00.281257 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.281223    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" containerID="2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d" exitCode=0
Sep 08 20:57:00.281442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.281255    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-8qknb" event={"ID":"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879","Type":"ContainerDied","Data":"2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d"}
Sep 08 20:57:00.281442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.281278    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-8qknb"
Sep 08 20:57:00.281442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.281313    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-8qknb" event={"ID":"ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879","Type":"ContainerDied","Data":"a22378e4d02f3d3eb9df1d791dca8672b8c03abedc6b94a625f66dd9d6f0e937"}
Sep 08 20:57:00.281442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.281335    2561 scope.go:117] "RemoveContainer" containerID="2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d"
Sep 08 20:57:00.291983 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.291937    2561 scope.go:117] "RemoveContainer" containerID="2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d"
Sep 08 20:57:00.292314 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:57:00.292293    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d\": container with ID starting with 2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d not found: ID does not exist" containerID="2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d"
Sep 08 20:57:00.292365 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.292325    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d"} err="failed to get container status \"2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d\": rpc error: code = NotFound desc = could not find container \"2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d\": container with ID starting with 2a61c5f14cb2bc71cad935db0e17590515b3818fb1d60d27ff34acc6a31ff15d not found: ID does not exist"
Sep 08 20:57:00.304860 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.304821    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-8qknb"]
Sep 08 20:57:00.309088 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.309055    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-8qknb"]
Sep 08 20:57:00.364005 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:00.363972    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" path="/var/lib/kubelet/pods/ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879/volumes"
Sep 08 20:57:01.155332 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155295    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-s2xw6"]
Sep 08 20:57:01.155750 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155735    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy"
Sep 08 20:57:01.155795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155752    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4009667b-5916-45cc-b2f6-ffe08e05b731" containerName="istio-proxy"
Sep 08 20:57:01.155795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155760    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy"
Sep 08 20:57:01.155795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155765    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ded1b8-b013-4f4a-9bee-4400c4e0e6e0" containerName="istio-proxy"
Sep 08 20:57:01.155795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155773    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy"
Sep 08 20:57:01.155795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155779    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5f2435a-d50c-484a-a9ca-b426bf19009d" containerName="istio-proxy"
Sep 08 20:57:01.155795 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155797    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" containerName="manager"
Sep 08 20:57:01.156034 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155803    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" containerName="manager"
Sep 08 20:57:01.156034 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.155869    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee0ae04e-8c27-4185-8bd9-0dfb7f2ea879" containerName="manager"
Sep 08 20:57:01.160409 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.160388    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.164431 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.164403    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 20:57:01.164823 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.164415    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 20:57:01.164991 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.164482    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 20:57:01.164991 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.164515    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qwqtd\""
Sep 08 20:57:01.169379 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.169198    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-s2xw6"]
Sep 08 20:57:01.239235 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.239190    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r2dn\" (UniqueName: \"kubernetes.io/projected/09758e51-c380-4215-b4ce-e5c125b960a9-kube-api-access-2r2dn\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.239424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.239268    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09758e51-c380-4215-b4ce-e5c125b960a9-webhook-cert\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.239424 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.239304    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09758e51-c380-4215-b4ce-e5c125b960a9-metrics-certs\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.340433 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.340389    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09758e51-c380-4215-b4ce-e5c125b960a9-webhook-cert\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.340626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.340447    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09758e51-c380-4215-b4ce-e5c125b960a9-metrics-certs\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.340626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.340513    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r2dn\" (UniqueName: \"kubernetes.io/projected/09758e51-c380-4215-b4ce-e5c125b960a9-kube-api-access-2r2dn\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.343258 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.343228    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09758e51-c380-4215-b4ce-e5c125b960a9-webhook-cert\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.343372 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.343288    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09758e51-c380-4215-b4ce-e5c125b960a9-metrics-certs\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.349237 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.349213    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r2dn\" (UniqueName: \"kubernetes.io/projected/09758e51-c380-4215-b4ce-e5c125b960a9-kube-api-access-2r2dn\") pod \"maas-controller-7cc45c6c68-s2xw6\" (UID: \"09758e51-c380-4215-b4ce-e5c125b960a9\") " pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.474705 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.474601    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:01.751846 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:01.751814    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cc45c6c68-s2xw6"]
Sep 08 20:57:01.753223 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:57:01.753193    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09758e51_c380_4215_b4ce_e5c125b960a9.slice/crio-ab0832983cd741ca51e9d4db540f5f65b01d9983d53656931edf2d2b71e228fa WatchSource:0}: Error finding container ab0832983cd741ca51e9d4db540f5f65b01d9983d53656931edf2d2b71e228fa: Status 404 returned error can't find the container with id ab0832983cd741ca51e9d4db540f5f65b01d9983d53656931edf2d2b71e228fa
Sep 08 20:57:02.296676 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:02.296636    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-s2xw6" event={"ID":"09758e51-c380-4215-b4ce-e5c125b960a9","Type":"ContainerStarted","Data":"ab0832983cd741ca51e9d4db540f5f65b01d9983d53656931edf2d2b71e228fa"}
Sep 08 20:57:03.307228 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:03.307193    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cc45c6c68-s2xw6" event={"ID":"09758e51-c380-4215-b4ce-e5c125b960a9","Type":"ContainerStarted","Data":"a4d419777d775009028e22ca44e3b7a3e2655bef8cbc5ae4df6e84b31f3c629b"}
Sep 08 20:57:03.307710 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:03.307261    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:03.327447 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:03.327382    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cc45c6c68-s2xw6" podStartSLOduration=1.672313308 podStartE2EDuration="2.327361836s" totalImagesPullingTime="655.048528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:57:03.324993023 +0000 UTC m=+1909.490163600" watchObservedRunningTime="2026-09-08 20:57:03.327361836 +0000 UTC m=+1909.492532411"
Sep 08 20:57:14.316306 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:14.316268    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cc45c6c68-s2xw6"
Sep 08 20:57:15.108265 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:15.108227    2561 scope.go:117] "RemoveContainer" containerID="3985e6bb78734442ba645b56cbaceb5ccc4886ab0a3a953e8136f301a553053e"
Sep 08 20:57:15.117086 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:57:15.117065    2561 scope.go:117] "RemoveContainer" containerID="b04a69bbce7ed645656e00563121613fea3f7dab58f4e16079546bc22f9ceded"
Sep 08 20:58:16.282763 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.282725    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"]
Sep 08 20:58:16.285130 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.282970    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr" podUID="d2e6f58d-3f17-4cc5-8164-06ed8d20b962" containerName="manager" containerID="cri-o://941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91" gracePeriod=10
Sep 08 20:58:16.545118 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.545061    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:58:16.664643 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.664590    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-extensions-socket-volume\") pod \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\" (UID: \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\") "
Sep 08 20:58:16.664832 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.664669    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8f85\" (UniqueName: \"kubernetes.io/projected/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-kube-api-access-v8f85\") pod \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\" (UID: \"d2e6f58d-3f17-4cc5-8164-06ed8d20b962\") "
Sep 08 20:58:16.665049 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.665027    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d2e6f58d-3f17-4cc5-8164-06ed8d20b962" (UID: "d2e6f58d-3f17-4cc5-8164-06ed8d20b962"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:58:16.666906 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.666873    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-kube-api-access-v8f85" (OuterVolumeSpecName: "kube-api-access-v8f85") pod "d2e6f58d-3f17-4cc5-8164-06ed8d20b962" (UID: "d2e6f58d-3f17-4cc5-8164-06ed8d20b962"). InnerVolumeSpecName "kube-api-access-v8f85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:58:16.758260 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.758226    2561 generic.go:356] "Generic (PLEG): container finished" podID="d2e6f58d-3f17-4cc5-8164-06ed8d20b962" containerID="941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91" exitCode=0
Sep 08 20:58:16.758442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.758272    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"
Sep 08 20:58:16.758442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.758306    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr" event={"ID":"d2e6f58d-3f17-4cc5-8164-06ed8d20b962","Type":"ContainerDied","Data":"941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91"}
Sep 08 20:58:16.758442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.758368    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr" event={"ID":"d2e6f58d-3f17-4cc5-8164-06ed8d20b962","Type":"ContainerDied","Data":"b111d430704f4368618f4dec17919f092bc4a849d18f590e45211c71843287b4"}
Sep 08 20:58:16.758442 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.758392    2561 scope.go:117] "RemoveContainer" containerID="941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91"
Sep 08 20:58:16.766242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.766216    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-extensions-socket-volume\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:58:16.766242 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.766242    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8f85\" (UniqueName: \"kubernetes.io/projected/d2e6f58d-3f17-4cc5-8164-06ed8d20b962-kube-api-access-v8f85\") on node \"ip-10-0-132-228.ec2.internal\" DevicePath \"\""
Sep 08 20:58:16.769560 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.769540    2561 scope.go:117] "RemoveContainer" containerID="941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91"
Sep 08 20:58:16.769998 ip-10-0-132-228 kubenswrapper[2561]: E0908 20:58:16.769871    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91\": container with ID starting with 941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91 not found: ID does not exist" containerID="941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91"
Sep 08 20:58:16.770067 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.770011    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91"} err="failed to get container status \"941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91\": rpc error: code = NotFound desc = could not find container \"941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91\": container with ID starting with 941591a3aa9da8782cc455f94c5dd36f0447121719e09081f822e59f4d376a91 not found: ID does not exist"
Sep 08 20:58:16.783802 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.783769    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"]
Sep 08 20:58:16.788495 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:16.788465    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tb5tr"]
Sep 08 20:58:18.363626 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:58:18.363589    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2e6f58d-3f17-4cc5-8164-06ed8d20b962" path="/var/lib/kubelet/pods/d2e6f58d-3f17-4cc5-8164-06ed8d20b962/volumes"
Sep 08 20:59:15.156486 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:15.156454    2561 scope.go:117] "RemoveContainer" containerID="9313b104d516556df3adfd9b1cb92b38fe4b86adc4b3c0c7a525ccc8c2ff13f4"
Sep 08 20:59:17.052193 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.052155    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"]
Sep 08 20:59:17.052637 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.052623    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e6f58d-3f17-4cc5-8164-06ed8d20b962" containerName="manager"
Sep 08 20:59:17.052693 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.052640    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e6f58d-3f17-4cc5-8164-06ed8d20b962" containerName="manager"
Sep 08 20:59:17.052732 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.052709    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e6f58d-3f17-4cc5-8164-06ed8d20b962" containerName="manager"
Sep 08 20:59:17.055753 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.055737    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:17.058552 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.058524    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xfsfx\""
Sep 08 20:59:17.077370 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.077340    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"]
Sep 08 20:59:17.134287 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.134253    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjmp9\" (UniqueName: \"kubernetes.io/projected/ccfde505-c4ea-4875-8c4a-696d07a04c46-kube-api-access-jjmp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w4h56\" (UID: \"ccfde505-c4ea-4875-8c4a-696d07a04c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:17.134470 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.134308    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccfde505-c4ea-4875-8c4a-696d07a04c46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w4h56\" (UID: \"ccfde505-c4ea-4875-8c4a-696d07a04c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:17.235710 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.235674    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccfde505-c4ea-4875-8c4a-696d07a04c46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w4h56\" (UID: \"ccfde505-c4ea-4875-8c4a-696d07a04c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:17.235930 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.235795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjmp9\" (UniqueName: \"kubernetes.io/projected/ccfde505-c4ea-4875-8c4a-696d07a04c46-kube-api-access-jjmp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w4h56\" (UID: \"ccfde505-c4ea-4875-8c4a-696d07a04c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:17.236098 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.236079    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccfde505-c4ea-4875-8c4a-696d07a04c46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w4h56\" (UID: \"ccfde505-c4ea-4875-8c4a-696d07a04c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:17.246890 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.246868    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjmp9\" (UniqueName: \"kubernetes.io/projected/ccfde505-c4ea-4875-8c4a-696d07a04c46-kube-api-access-jjmp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w4h56\" (UID: \"ccfde505-c4ea-4875-8c4a-696d07a04c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:17.366147 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.366060    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:17.652987 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:17.652961    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"]
Sep 08 20:59:17.655025 ip-10-0-132-228 kubenswrapper[2561]: W0908 20:59:17.654999    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccfde505_c4ea_4875_8c4a_696d07a04c46.slice/crio-1cfb6c3ce4180e515af840f248318daf7857c92c94973a80593d01a151cb6a61 WatchSource:0}: Error finding container 1cfb6c3ce4180e515af840f248318daf7857c92c94973a80593d01a151cb6a61: Status 404 returned error can't find the container with id 1cfb6c3ce4180e515af840f248318daf7857c92c94973a80593d01a151cb6a61
Sep 08 20:59:18.119150 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:18.119056    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56" event={"ID":"ccfde505-c4ea-4875-8c4a-696d07a04c46","Type":"ContainerStarted","Data":"a99a230a6cf8ed45f8948705a3273161dc249a40988cc7a9dc9489b8abb727cb"}
Sep 08 20:59:18.119150 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:18.119115    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56" event={"ID":"ccfde505-c4ea-4875-8c4a-696d07a04c46","Type":"ContainerStarted","Data":"1cfb6c3ce4180e515af840f248318daf7857c92c94973a80593d01a151cb6a61"}
Sep 08 20:59:18.119628 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:18.119156    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 20:59:18.154973 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:18.154920    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56" podStartSLOduration=1.154889076 podStartE2EDuration="1.154889076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:59:18.152941313 +0000 UTC m=+2044.318111889" watchObservedRunningTime="2026-09-08 20:59:18.154889076 +0000 UTC m=+2044.320059652"
Sep 08 20:59:29.128509 ip-10-0-132-228 kubenswrapper[2561]: I0908 20:59:29.128476    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w4h56"
Sep 08 21:00:00.152912 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:00.152868    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j"]
Sep 08 21:00:00.156956 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:00.156931    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j"
Sep 08 21:00:00.163577 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:00.163540    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 21:00:00.166930 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:00.166909    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j"
Sep 08 21:00:00.175170 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:00.175141    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j"]
Sep 08 21:00:00.453013 ip-10-0-132-228 kubenswrapper[2561]: W0908 21:00:00.452971    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6ada6d_64dd_4252_815d_60c3ef0e972a.slice/crio-09f22723e525f890f2b9d8982c6aae3a6046254a1e098dbfcfab1badbf80be83 WatchSource:0}: Error finding container 09f22723e525f890f2b9d8982c6aae3a6046254a1e098dbfcfab1badbf80be83: Status 404 returned error can't find the container with id 09f22723e525f890f2b9d8982c6aae3a6046254a1e098dbfcfab1badbf80be83
Sep 08 21:00:00.453395 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:00.453368    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j"]
Sep 08 21:00:01.389722 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:01.389684    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j" event={"ID":"8b6ada6d-64dd-4252-815d-60c3ef0e972a","Type":"ContainerStarted","Data":"57779c103713f84669429d66c403bc90a746dac1ce62a5edbde315ec605618f3"}
Sep 08 21:00:01.390159 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:01.389735    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j" event={"ID":"8b6ada6d-64dd-4252-815d-60c3ef0e972a","Type":"ContainerStarted","Data":"09f22723e525f890f2b9d8982c6aae3a6046254a1e098dbfcfab1badbf80be83"}
Sep 08 21:00:01.409799 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:01.409738    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j" podStartSLOduration=1.409720655 podStartE2EDuration="1.409720655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 21:00:01.40858253 +0000 UTC m=+2087.573753117" watchObservedRunningTime="2026-09-08 21:00:01.409720655 +0000 UTC m=+2087.574891230"
Sep 08 21:00:02.397163 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:02.397123    2561 generic.go:356] "Generic (PLEG): container finished" podID="8b6ada6d-64dd-4252-815d-60c3ef0e972a" containerID="57779c103713f84669429d66c403bc90a746dac1ce62a5edbde315ec605618f3" exitCode=0
Sep 08 21:00:02.397607 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:02.397209    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j" event={"ID":"8b6ada6d-64dd-4252-815d-60c3ef0e972a","Type":"ContainerDied","Data":"57779c103713f84669429d66c403bc90a746dac1ce62a5edbde315ec605618f3"}
Sep 08 21:00:03.547161 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:03.547136    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j"
Sep 08 21:00:04.410526 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:04.410500    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j"
Sep 08 21:00:04.410702 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:04.410498    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815020-57l6j" event={"ID":"8b6ada6d-64dd-4252-815d-60c3ef0e972a","Type":"ContainerDied","Data":"09f22723e525f890f2b9d8982c6aae3a6046254a1e098dbfcfab1badbf80be83"}
Sep 08 21:00:04.410702 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:04.410625    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09f22723e525f890f2b9d8982c6aae3a6046254a1e098dbfcfab1badbf80be83"
Sep 08 21:00:14.600886 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:14.600855    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 21:00:14.608404 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:14.608377    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 21:00:15.186452 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:00:15.186417    2561 scope.go:117] "RemoveContainer" containerID="918e59dafdb17b48d30e8ddf1f37540c560343d215575d0a1443437f9a42f52d"
Sep 08 21:01:26.763789 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:26.763704    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4ljhv_5c899c43-da7b-4143-bd29-86ef5795a956/manager/0.log"
Sep 08 21:01:26.882937 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:26.882908    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pp25j_0bd25186-ab2f-4cc8-b1b8-cad530122972/manager/0.log"
Sep 08 21:01:27.011162 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:27.011125    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7cc45c6c68-s2xw6_09758e51-c380-4215-b4ce-e5c125b960a9/manager/0.log"
Sep 08 21:01:27.496386 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:27.496360    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbd9f8d48-bkbz5_71846c51-60c3-4773-b2e9-6f44d404c6e2/copy-manifests/0.log"
Sep 08 21:01:27.509955 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:27.509922    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbd9f8d48-bkbz5_71846c51-60c3-4773-b2e9-6f44d404c6e2/manager/0.log"
Sep 08 21:01:28.211318 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:28.211288    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29815005-jd4km_5ec337b6-07d9-46c9-af42-e90d346ad8ef/cleanup/0.log"
Sep 08 21:01:28.324462 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:28.324430    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29815020-57l6j_8b6ada6d-64dd-4252-815d-60c3ef0e972a/cleanup/0.log"
Sep 08 21:01:28.458153 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:28.458124    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-p59dk_2ab28082-7dda-4daa-943c-7679b5d8b950/postgres/0.log"
Sep 08 21:01:29.506329 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:29.506287    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-q8879_50eb3405-5e3c-44b5-b81b-34afbacf7bbf/manager/0.log"
Sep 08 21:01:29.651426 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:29.651381    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-tqflh_09b25262-3a59-4ad5-8767-b30f3198e5c5/manager/0.log"
Sep 08 21:01:29.784862 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:29.784776    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-tnlpb_1ed40483-8bef-4048-aec9-722a9869e33c/kuadrant-console-plugin/0.log"
Sep 08 21:01:29.934925 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:29.934314    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-w4h56_ccfde505-c4ea-4875-8c4a-696d07a04c46/manager/0.log"
Sep 08 21:01:30.190208 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:30.190176    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-jl79h_19e6dd31-06df-4f05-83b2-4efe18f97aef/manager/0.log"
Sep 08 21:01:30.943638 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:30.943608    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bf757d869-sddql_6b6950c4-add4-4969-8d1a-bfcbd268f02c/kube-auth-proxy/0.log"
Sep 08 21:01:31.067790 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:31.067756    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kv8wj_ddd858c1-029d-4d85-ac36-dce67c4f4ebf/istio-proxy/1.log"
Sep 08 21:01:31.194703 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:31.194615    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rv2lp_b2e50305-6c7a-4b87-8f30-fe9e06b6c367/payload-pre-processing/0.log"
Sep 08 21:01:31.338847 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:31.338791    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mw9v6_e7b6128d-c751-4948-8312-f0ed78e55355/payload-processing/0.log"
Sep 08 21:01:31.470879 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:31.470798    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c98988766-lv6fl_350750d6-ab60-46cc-9421-cdb689c07574/router/0.log"
Sep 08 21:01:32.113406 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:32.113377    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-rdgqx_4b4cc211-d382-4cf9-b137-3b204967776a/main/0.log"
Sep 08 21:01:38.501791 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.501752    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w75tg/must-gather-l5zgv"]
Sep 08 21:01:38.502340 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.502246    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b6ada6d-64dd-4252-815d-60c3ef0e972a" containerName="cleanup"
Sep 08 21:01:38.502340 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.502262    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b6ada6d-64dd-4252-815d-60c3ef0e972a" containerName="cleanup"
Sep 08 21:01:38.502413 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.502357    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b6ada6d-64dd-4252-815d-60c3ef0e972a" containerName="cleanup"
Sep 08 21:01:38.505874 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.505853    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w75tg/must-gather-l5zgv"
Sep 08 21:01:38.510770 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.510736    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w75tg\"/\"kube-root-ca.crt\""
Sep 08 21:01:38.511332 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.511311    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w75tg\"/\"default-dockercfg-xmph6\""
Sep 08 21:01:38.511460 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.511404    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w75tg\"/\"openshift-service-ca.crt\""
Sep 08 21:01:38.515719 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.515667    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg457\" (UniqueName: \"kubernetes.io/projected/6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0-kube-api-access-qg457\") pod \"must-gather-l5zgv\" (UID: \"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0\") " pod="openshift-must-gather-w75tg/must-gather-l5zgv"
Sep 08 21:01:38.515861 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.515836    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0-must-gather-output\") pod \"must-gather-l5zgv\" (UID: \"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0\") " pod="openshift-must-gather-w75tg/must-gather-l5zgv"
Sep 08 21:01:38.517196 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.517175    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w75tg/must-gather-l5zgv"]
Sep 08 21:01:38.617089 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.617057    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0-must-gather-output\") pod \"must-gather-l5zgv\" (UID: \"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0\") " pod="openshift-must-gather-w75tg/must-gather-l5zgv"
Sep 08 21:01:38.617265 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.617110    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg457\" (UniqueName: \"kubernetes.io/projected/6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0-kube-api-access-qg457\") pod \"must-gather-l5zgv\" (UID: \"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0\") " pod="openshift-must-gather-w75tg/must-gather-l5zgv"
Sep 08 21:01:38.617383 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.617361    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0-must-gather-output\") pod \"must-gather-l5zgv\" (UID: \"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0\") " pod="openshift-must-gather-w75tg/must-gather-l5zgv"
Sep 08 21:01:38.626947 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.626886    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg457\" (UniqueName: \"kubernetes.io/projected/6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0-kube-api-access-qg457\") pod \"must-gather-l5zgv\" (UID: \"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0\") " pod="openshift-must-gather-w75tg/must-gather-l5zgv"
Sep 08 21:01:38.817319 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:38.817219    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w75tg/must-gather-l5zgv"
Sep 08 21:01:39.117757 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:39.117730    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w75tg/must-gather-l5zgv"]
Sep 08 21:01:39.120063 ip-10-0-132-228 kubenswrapper[2561]: W0908 21:01:39.120025    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b1e8a05_1ffd_4523_b1f4_49b4062a8bd0.slice/crio-43fc3b413ede5d1f3544470a840150acedd21942fed1e5eb87ddc852cd722f1f WatchSource:0}: Error finding container 43fc3b413ede5d1f3544470a840150acedd21942fed1e5eb87ddc852cd722f1f: Status 404 returned error can't find the container with id 43fc3b413ede5d1f3544470a840150acedd21942fed1e5eb87ddc852cd722f1f
Sep 08 21:01:39.121964 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:39.121946    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 21:01:39.996850 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:39.996817    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w75tg/must-gather-l5zgv" event={"ID":"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0","Type":"ContainerStarted","Data":"a19b44693dfde7e5f92f7d1d7c8769f1b6c8428bb8e143f662027ea099cfa43f"}
Sep 08 21:01:39.997321 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:39.996862    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w75tg/must-gather-l5zgv" event={"ID":"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0","Type":"ContainerStarted","Data":"43fc3b413ede5d1f3544470a840150acedd21942fed1e5eb87ddc852cd722f1f"}
Sep 08 21:01:41.009016 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:41.008971    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w75tg/must-gather-l5zgv" event={"ID":"6b1e8a05-1ffd-4523-b1f4-49b4062a8bd0","Type":"ContainerStarted","Data":"69a09410f20b1d9b6f9d0d636b02dc509198b06fafdb6df83ff313a269a1f4e5"}
Sep 08 21:01:41.029143 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:41.029072    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-w75tg/must-gather-l5zgv" podStartSLOduration=2.272184438 podStartE2EDuration="3.029050333s" totalImagesPullingTime="756.865895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 21:01:41.026828184 +0000 UTC m=+2187.191998760" watchObservedRunningTime="2026-09-08 21:01:41.029050333 +0000 UTC m=+2187.194220910"
Sep 08 21:01:41.560312 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:41.560283    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rqs79_92d5f074-5b53-43a6-b31e-5136411eba01/global-pull-secret-syncer/0.log"
Sep 08 21:01:41.644589 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:41.644552    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vnw88_b6a41838-e8f5-4756-be48-ff1b26dcf96a/konnectivity-agent/0.log"
Sep 08 21:01:41.734211 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:41.734180    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-228.ec2.internal_dd93909c83a29abfbb61422c635288b0/haproxy/0.log"
Sep 08 21:01:46.247075 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:46.246968    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-q8879_50eb3405-5e3c-44b5-b81b-34afbacf7bbf/manager/0.log"
Sep 08 21:01:46.277354 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:46.277221    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-tqflh_09b25262-3a59-4ad5-8767-b30f3198e5c5/manager/0.log"
Sep 08 21:01:46.307919 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:46.307845    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-tnlpb_1ed40483-8bef-4048-aec9-722a9869e33c/kuadrant-console-plugin/0.log"
Sep 08 21:01:46.354661 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:46.354597    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-w4h56_ccfde505-c4ea-4875-8c4a-696d07a04c46/manager/0.log"
Sep 08 21:01:46.462075 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:46.461351    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-jl79h_19e6dd31-06df-4f05-83b2-4efe18f97aef/manager/0.log"
Sep 08 21:01:47.938946 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:47.938884    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_badbccaa-859d-4250-b6cc-64f9997152e2/alertmanager/0.log"
Sep 08 21:01:47.968609 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:47.968536    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_badbccaa-859d-4250-b6cc-64f9997152e2/config-reloader/0.log"
Sep 08 21:01:47.994354 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:47.994326    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_badbccaa-859d-4250-b6cc-64f9997152e2/kube-rbac-proxy-web/0.log"
Sep 08 21:01:48.019248 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.019192    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_badbccaa-859d-4250-b6cc-64f9997152e2/kube-rbac-proxy/0.log"
Sep 08 21:01:48.052268 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.052201    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_badbccaa-859d-4250-b6cc-64f9997152e2/kube-rbac-proxy-metric/0.log"
Sep 08 21:01:48.074004 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.073977    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_badbccaa-859d-4250-b6cc-64f9997152e2/prom-label-proxy/0.log"
Sep 08 21:01:48.096652 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.096584    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_badbccaa-859d-4250-b6cc-64f9997152e2/init-config-reloader/0.log"
Sep 08 21:01:48.183088 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.183043    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-cdmjn_a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07/kube-state-metrics/0.log"
Sep 08 21:01:48.211643 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.211521    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-cdmjn_a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07/kube-rbac-proxy-main/0.log"
Sep 08 21:01:48.237107 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.237057    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-cdmjn_a80cc9ba-7fec-4b0d-b5cd-2b4cec6acc07/kube-rbac-proxy-self/0.log"
Sep 08 21:01:48.441993 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.441964    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vmw6j_b6b0599e-1af2-48a8-9171-1777ac9074b2/node-exporter/0.log"
Sep 08 21:01:48.468788 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.468708    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vmw6j_b6b0599e-1af2-48a8-9171-1777ac9074b2/kube-rbac-proxy/0.log"
Sep 08 21:01:48.502727 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.502699    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vmw6j_b6b0599e-1af2-48a8-9171-1777ac9074b2/init-textfile/0.log"
Sep 08 21:01:48.757656 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.757622    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-7xffg_e56dd66b-f18a-4d3c-a71a-b06ae1204c7a/kube-rbac-proxy-main/0.log"
Sep 08 21:01:48.836561 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.836533    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-7xffg_e56dd66b-f18a-4d3c-a71a-b06ae1204c7a/kube-rbac-proxy-self/0.log"
Sep 08 21:01:48.874998 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:48.874969    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-7xffg_e56dd66b-f18a-4d3c-a71a-b06ae1204c7a/openshift-state-metrics/0.log"
Sep 08 21:01:49.095600 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:49.095474    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-4d542_48d57a6b-d406-4864-924d-bea5515378f1/prometheus-operator/0.log"
Sep 08 21:01:49.122615 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:49.122571    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-4d542_48d57a6b-d406-4864-924d-bea5515378f1/kube-rbac-proxy/0.log"
Sep 08 21:01:49.198209 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:49.198120    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69cbfcd64f-fbdcr_476785d2-a0aa-4fe9-8ac1-e0070e9d12bd/telemeter-client/0.log"
Sep 08 21:01:49.224382 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:49.224313    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69cbfcd64f-fbdcr_476785d2-a0aa-4fe9-8ac1-e0070e9d12bd/reload/0.log"
Sep 08 21:01:49.255476 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:49.255446    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69cbfcd64f-fbdcr_476785d2-a0aa-4fe9-8ac1-e0070e9d12bd/kube-rbac-proxy/0.log"
Sep 08 21:01:50.868000 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:50.867968    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-tnls4_7d8ba035-e6a4-4e15-9a1e-79b7e6742b41/networking-console-plugin/0.log"
Sep 08 21:01:51.580646 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:51.580587    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/1.log"
Sep 08 21:01:51.589374 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:51.589335    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-h4c88_be92c86c-f31f-43e5-94a5-66d0d8dc4415/console-operator/2.log"
Sep 08 21:01:52.010688 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.010647    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"]
Sep 08 21:01:52.017133 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.017100    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.022237 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.022200    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"]
Sep 08 21:01:52.065134 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.065094    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-podres\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.065339 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.065170    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c88rn\" (UniqueName: \"kubernetes.io/projected/28a2a625-5bf9-46ca-b882-00fea322b348-kube-api-access-c88rn\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.065339 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.065303    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-sys\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.065339 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.065336    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-lib-modules\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.065535 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.065398    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-proc\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167150 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167104    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-sys\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167367 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167173    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-lib-modules\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167367 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167226    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-proc\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167367 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167217    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-sys\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167367 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167320    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-podres\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167367 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167347    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c88rn\" (UniqueName: \"kubernetes.io/projected/28a2a625-5bf9-46ca-b882-00fea322b348-kube-api-access-c88rn\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167598 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167399    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-lib-modules\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167598 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167324    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-proc\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.167598 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.167478    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28a2a625-5bf9-46ca-b882-00fea322b348-podres\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.176081 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.176048    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c88rn\" (UniqueName: \"kubernetes.io/projected/28a2a625-5bf9-46ca-b882-00fea322b348-kube-api-access-c88rn\") pod \"perf-node-gather-daemonset-rpfgx\" (UID: \"28a2a625-5bf9-46ca-b882-00fea322b348\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.176773 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.176751    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c4c56559d-xbb8q_e9cbbb9e-677a-4e46-88c9-dd500e8dd92d/console/0.log"
Sep 08 21:01:52.217373 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.217285    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-bc4cl_2b29a176-eaab-4b02-83fd-1334280c7b3d/download-server/0.log"
Sep 08 21:01:52.337441 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.337156    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:52.633535 ip-10-0-132-228 kubenswrapper[2561]: W0908 21:01:52.633485    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod28a2a625_5bf9_46ca_b882_00fea322b348.slice/crio-c718aa0aefc00d8af77cee45f04f0c1f2ea1dd49c93b89d98d3a4fcb999f0d00 WatchSource:0}: Error finding container c718aa0aefc00d8af77cee45f04f0c1f2ea1dd49c93b89d98d3a4fcb999f0d00: Status 404 returned error can't find the container with id c718aa0aefc00d8af77cee45f04f0c1f2ea1dd49c93b89d98d3a4fcb999f0d00
Sep 08 21:01:52.634588 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:52.634557    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"]
Sep 08 21:01:53.141521 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:53.141476    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx" event={"ID":"28a2a625-5bf9-46ca-b882-00fea322b348","Type":"ContainerStarted","Data":"98f304cdbc7736ab40f29f16596e22036f91c659db1c06abec44184a9da16113"}
Sep 08 21:01:53.142017 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:53.141584    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx" event={"ID":"28a2a625-5bf9-46ca-b882-00fea322b348","Type":"ContainerStarted","Data":"c718aa0aefc00d8af77cee45f04f0c1f2ea1dd49c93b89d98d3a4fcb999f0d00"}
Sep 08 21:01:53.142017 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:53.141625    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:01:53.160918 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:53.158361    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx" podStartSLOduration=2.158337688 podStartE2EDuration="2.158337688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 21:01:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 21:01:53.156202806 +0000 UTC m=+2199.321373383" watchObservedRunningTime="2026-09-08 21:01:53.158337688 +0000 UTC m=+2199.323508265"
Sep 08 21:01:54.167197 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:54.167170    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xmz24_68bd9156-10b5-407c-8ef9-3ce4d0e0204b/dns/0.log"
Sep 08 21:01:54.186175 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:54.186148    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xmz24_68bd9156-10b5-407c-8ef9-3ce4d0e0204b/kube-rbac-proxy/0.log"
Sep 08 21:01:54.229257 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:54.229229    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-psrxw_f938a669-effc-4bb0-8894-fdb5be304643/dns-node-resolver/0.log"
Sep 08 21:01:54.828318 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:54.828290    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sktlj_2ebae549-d876-40d4-a52c-025f6b21dde0/node-ca/0.log"
Sep 08 21:01:56.245844 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:56.245811    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bf757d869-sddql_6b6950c4-add4-4969-8d1a-bfcbd268f02c/kube-auth-proxy/0.log"
Sep 08 21:01:56.278473 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:56.278435    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kv8wj_ddd858c1-029d-4d85-ac36-dce67c4f4ebf/istio-proxy/0.log"
Sep 08 21:01:56.288777 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:56.288748    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kv8wj_ddd858c1-029d-4d85-ac36-dce67c4f4ebf/istio-proxy/1.log"
Sep 08 21:01:56.325709 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:56.325680    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rv2lp_b2e50305-6c7a-4b87-8f30-fe9e06b6c367/payload-pre-processing/0.log"
Sep 08 21:01:56.365018 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:56.364979    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mw9v6_e7b6128d-c751-4948-8312-f0ed78e55355/payload-processing/0.log"
Sep 08 21:01:56.389410 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:56.389381    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c98988766-lv6fl_350750d6-ab60-46cc-9421-cdb689c07574/router/0.log"
Sep 08 21:01:57.284912 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:57.284870    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j69nw_74e19476-d6be-47fc-a532-4129d15638af/serve-healthcheck-canary/0.log"
Sep 08 21:01:57.873752 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:57.873723    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kjrd7_bbcd2699-0b1b-4611-96dc-18129e3364f8/kube-rbac-proxy/0.log"
Sep 08 21:01:57.893029 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:57.893001    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kjrd7_bbcd2699-0b1b-4611-96dc-18129e3364f8/exporter/0.log"
Sep 08 21:01:57.913197 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:57.913166    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kjrd7_bbcd2699-0b1b-4611-96dc-18129e3364f8/extractor/0.log"
Sep 08 21:01:59.159028 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:01:59.158997    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-rpfgx"
Sep 08 21:02:00.108351 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:00.108316    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4ljhv_5c899c43-da7b-4143-bd29-86ef5795a956/manager/0.log"
Sep 08 21:02:00.351802 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:00.351770    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pp25j_0bd25186-ab2f-4cc8-b1b8-cad530122972/manager/0.log"
Sep 08 21:02:00.409796 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:00.409749    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7cc45c6c68-s2xw6_09758e51-c380-4215-b4ce-e5c125b960a9/manager/0.log"
Sep 08 21:02:00.642093 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:00.642026    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbd9f8d48-bkbz5_71846c51-60c3-4773-b2e9-6f44d404c6e2/manager/0.log"
Sep 08 21:02:00.659945 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:00.659835    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbd9f8d48-bkbz5_71846c51-60c3-4773-b2e9-6f44d404c6e2/copy-manifests/0.log"
Sep 08 21:02:03.210516 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:03.210481    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bf757d869-sddql_6b6950c4-add4-4969-8d1a-bfcbd268f02c/kube-auth-proxy/0.log"
Sep 08 21:02:03.233868 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:03.233832    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kv8wj_ddd858c1-029d-4d85-ac36-dce67c4f4ebf/istio-proxy/0.log"
Sep 08 21:02:03.246261 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:03.246225    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kv8wj_ddd858c1-029d-4d85-ac36-dce67c4f4ebf/istio-proxy/1.log"
Sep 08 21:02:03.275166 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:03.275111    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rv2lp_b2e50305-6c7a-4b87-8f30-fe9e06b6c367/payload-pre-processing/0.log"
Sep 08 21:02:03.308025 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:03.307989    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mw9v6_e7b6128d-c751-4948-8312-f0ed78e55355/payload-processing/0.log"
Sep 08 21:02:03.324558 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:03.324530    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c98988766-lv6fl_350750d6-ab60-46cc-9421-cdb689c07574/router/0.log"
Sep 08 21:02:05.226493 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:05.226464    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bf757d869-sddql_6b6950c4-add4-4969-8d1a-bfcbd268f02c/kube-auth-proxy/0.log"
Sep 08 21:02:05.253150 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:05.253119    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kv8wj_ddd858c1-029d-4d85-ac36-dce67c4f4ebf/istio-proxy/0.log"
Sep 08 21:02:05.265805 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:05.265775    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kv8wj_ddd858c1-029d-4d85-ac36-dce67c4f4ebf/istio-proxy/1.log"
Sep 08 21:02:05.301285 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:05.301214    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rv2lp_b2e50305-6c7a-4b87-8f30-fe9e06b6c367/payload-pre-processing/0.log"
Sep 08 21:02:05.332483 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:05.332452    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-mw9v6_e7b6128d-c751-4948-8312-f0ed78e55355/payload-processing/0.log"
Sep 08 21:02:05.355642 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:05.355607    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c98988766-lv6fl_350750d6-ab60-46cc-9421-cdb689c07574/router/0.log"
Sep 08 21:02:07.708230 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:07.708185    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-fl5q8_a9733443-6da4-4bd0-99c1-0115e2aa4623/kube-storage-version-migrator-operator/1.log"
Sep 08 21:02:07.708999 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:07.708974    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-fl5q8_a9733443-6da4-4bd0-99c1-0115e2aa4623/kube-storage-version-migrator-operator/0.log"
Sep 08 21:02:08.813583 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:08.813559    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4vfsr_ee07fbc3-149f-43cb-8d58-4cc6aef52bce/kube-multus-additional-cni-plugins/0.log"
Sep 08 21:02:08.834981 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:08.834937    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4vfsr_ee07fbc3-149f-43cb-8d58-4cc6aef52bce/egress-router-binary-copy/0.log"
Sep 08 21:02:08.857852 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:08.857829    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4vfsr_ee07fbc3-149f-43cb-8d58-4cc6aef52bce/cni-plugins/0.log"
Sep 08 21:02:08.879627 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:08.879553    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4vfsr_ee07fbc3-149f-43cb-8d58-4cc6aef52bce/bond-cni-plugin/0.log"
Sep 08 21:02:08.900819 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:08.900774    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4vfsr_ee07fbc3-149f-43cb-8d58-4cc6aef52bce/routeoverride-cni/0.log"
Sep 08 21:02:08.922822 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:08.922791    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4vfsr_ee07fbc3-149f-43cb-8d58-4cc6aef52bce/whereabouts-cni-bincopy/0.log"
Sep 08 21:02:08.948953 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:08.948926    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4vfsr_ee07fbc3-149f-43cb-8d58-4cc6aef52bce/whereabouts-cni/0.log"
Sep 08 21:02:09.353342 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:09.353311    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gtfhv_97cb2286-ef93-4a2d-bb42-0ec62ab4e2a1/kube-multus/0.log"
Sep 08 21:02:09.380209 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:09.380180    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qxhpg_f559af35-ddd5-4034-93fb-94b0e072bbd0/network-metrics-daemon/0.log"
Sep 08 21:02:09.400019 ip-10-0-132-228 kubenswrapper[2561]: I0908 21:02:09.399991    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qxhpg_f559af35-ddd5-4034-93fb-94b0e072bbd0/kube-rbac-proxy/0.log"