Sep 15 12:44:42.086344 ip-10-0-132-169 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 12:44:42.540606 ip-10-0-132-169 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 12:44:42.540606 ip-10-0-132-169 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 12:44:42.540606 ip-10-0-132-169 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 12:44:42.540606 ip-10-0-132-169 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 12:44:42.540606 ip-10-0-132-169 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 12:44:42.541557 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.541472    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 12:44:42.545219 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545204    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:44:42.545219 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545219    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545225    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545228    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545231    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545234    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545237    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545240    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545243    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545249    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545252    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545254    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545257    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545260    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545262    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545267    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545270    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545273    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545276    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545278    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:44:42.545279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545281    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545284    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545287    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545290    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545293    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545295    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545298    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545301    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545305    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545309    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545312    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545314    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545317    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545320    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545323    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545326    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545329    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545331    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545335    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:44:42.545781 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545337    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545340    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545342    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545345    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545347    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545350    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545353    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545355    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545357    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545360    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545362    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545365    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545367    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545370    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545372    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545376    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545379    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545382    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545384    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545387    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:44:42.546581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545390    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545392    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545395    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545397    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545399    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545402    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545405    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545407    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545410    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545412    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545415    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545417    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545420    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545422    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545425    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545427    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545430    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545433    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545435    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545439    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:44:42.547097 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545441    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545445    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545448    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545450    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545452    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545455    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545458    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545461    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545464    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545466    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545470    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545473    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545476    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545479    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545481    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545484    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545486    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545489    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545492    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.545494    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:44:42.547666 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547160    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547172    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547178    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547181    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547184    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547187    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547190    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547193    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547195    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547198    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547201    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547204    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547206    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547210    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547215    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547218    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547221    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547224    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547226    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:44:42.548164 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547229    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547232    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547235    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547237    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547240    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547243    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547247    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547250    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547252    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547255    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547257    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547260    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547262    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547265    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547267    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547270    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547273    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547276    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547278    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547281    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:44:42.548644 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547283    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547286    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547288    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547291    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547293    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547296    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547299    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547303    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547305    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547308    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547311    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547314    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547317    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547320    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547322    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547325    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547327    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547330    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547332    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547335    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:44:42.549141 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547337    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547345    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547347    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547350    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547352    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547355    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547358    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547360    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547363    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547365    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547368    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547371    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547373    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547376    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547379    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547381    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547390    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547393    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547396    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:44:42.549635 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547401    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547404    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547408    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547411    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547414    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547421    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547423    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547426    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547429    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547431    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547434    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547436    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547438    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547441    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547444    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547446    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547449    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547451    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547454    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547457    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:44:42.550121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547459    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547835    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547840    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547843    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547846    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547849    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547851    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547854    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547857    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547860    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547862    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547865    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547869    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547871    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547874    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547877    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547880    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547883    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547885    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547888    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:44:42.550624 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547891    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547893    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547896    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547899    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547901    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547904    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547906    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547909    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547913    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547916    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547919    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547922    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547925    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547930    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547934    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547937    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547940    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547943    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547946    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:44:42.551121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547949    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547951    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547954    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547957    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547960    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547962    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547965    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547967    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547970    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547972    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547975    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547985    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547988    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547990    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547992    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547995    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.547998    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548000    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548003    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:44:42.551581 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548006    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548008    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548011    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548013    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548016    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548019    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548021    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548024    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548039    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548042    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548046    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548048    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548051    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548054    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548056    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548059    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548061    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548064    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548067    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548069    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:44:42.552094 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548072    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548074    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548077    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548080    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548083    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548086    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548088    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548091    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548093    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548096    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548098    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548101    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548104    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548106    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548109    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548111    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548114    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548116    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548119    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548121    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:44:42.552583 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548126    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548128    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548194    2562 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548232    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548240    2562 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548245    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548249    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548252    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548256    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548260    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548263    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548266    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548269    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548272    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548275    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548278    2562 flags.go:64] FLAG: --cgroup-root=""
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548281    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548284    2562 flags.go:64] FLAG: --client-ca-file=""
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548288    2562 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548291    2562 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548295    2562 flags.go:64] FLAG: --cluster-domain=""
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548298    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548301    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548304    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 12:44:42.553071 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548307    2562 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548311    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548314    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548317    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548320    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548325    2562 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548328    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548331    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548334    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548337    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548341    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548344    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548347    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548350    2562 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548353    2562 flags.go:64] FLAG: --enable-server="true"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548356    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548360    2562 flags.go:64] FLAG: --event-burst="100"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548363    2562 flags.go:64] FLAG: --event-qps="50"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548366    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548369    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548372    2562 flags.go:64] FLAG: --eviction-hard=""
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548375    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548378    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548381    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548384    2562 flags.go:64] FLAG: --eviction-soft=""
Sep 15 12:44:42.553672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548387    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548390    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548393    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548396    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548399    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548402    2562 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548405    2562 flags.go:64] FLAG: --feature-gates=""
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548409    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548412    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548415    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548418    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548421    2562 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548425    2562 flags.go:64] FLAG: --help="false"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548428    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-169.ec2.internal"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548431    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548434    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548437    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548440    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548444    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548446    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548449    2562 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548452    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548455    2562 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 12:44:42.554313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548457    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548460    2562 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548463    2562 flags.go:64] FLAG: --kube-reserved=""
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548466    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548469    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548472    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548474    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548477    2562 flags.go:64] FLAG: --lock-file=""
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548480    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548482    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548485    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548491    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548494    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548497    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548500    2562 flags.go:64] FLAG: --logging-format="text"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548503    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548506    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548509    2562 flags.go:64] FLAG: --manifest-url=""
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548512    2562 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548515    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548518    2562 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548523    2562 flags.go:64] FLAG: --max-pods="110"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548526    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548530    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548533    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 12:44:42.554902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548535    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548538    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548541    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548544    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548551    2562 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548554    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548557    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548560    2562 flags.go:64] FLAG: --pod-cidr=""
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548563    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548568    2562 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548570    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548573    2562 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548576    2562 flags.go:64] FLAG: --port="10250"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548579    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548582    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f011d61ec59271d"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548585    2562 flags.go:64] FLAG: --qos-reserved=""
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548588    2562 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548591    2562 flags.go:64] FLAG: --register-node="true"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548594    2562 flags.go:64] FLAG: --register-with-taints=""
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548597    2562 flags.go:64] FLAG: --registry-burst="10"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548601    2562 flags.go:64] FLAG: --registry-qps="5"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548604    2562 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548607    2562 flags.go:64] FLAG: --reserved-memory=""
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548610    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548613    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 12:44:42.555562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548616    2562 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548619    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548621    2562 flags.go:64] FLAG: --runonce="false"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548624    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548629    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548631    2562 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548634    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548637    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548640    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548643    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548646    2562 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548649    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548652    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548655    2562 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548658    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548660    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548663    2562 flags.go:64] FLAG: --system-cgroups=""
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548666    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548671    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548674    2562 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548677    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548682    2562 flags.go:64] FLAG: --tls-min-version=""
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548685    2562 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548688    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548690    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 12:44:42.556201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548693    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548696    2562 flags.go:64] FLAG: --v="2"
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548704    2562 flags.go:64] FLAG: --version="false"
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548708    2562 flags.go:64] FLAG: --vmodule=""
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548712    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.548716    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548845    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548849    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548852    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548854    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548857    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548861    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548863    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548868    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548870    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548873    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548875    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548878    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548881    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548883    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548886    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:44:42.556800 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548888    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548891    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548893    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548896    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548898    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548901    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548903    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548906    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548909    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548911    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548915    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548919    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548922    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548926    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548929    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548933    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548936    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548938    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548941    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:44:42.557365 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548945    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548948    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548950    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548954    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548957    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548960    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548963    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548966    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548968    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548971    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548973    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548976    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548979    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548981    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548984    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548986    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548989    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548992    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548995    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.548999    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:44:42.557825 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549002    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549006    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549010    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549012    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549015    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549017    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549020    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549023    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549026    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549041    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549044    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549046    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549050    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549052    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549055    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549060    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549062    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549066    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549069    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549072    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:44:42.558323 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549074    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549078    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549081    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549084    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549087    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549090    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549092    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549095    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549098    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549101    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549104    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549106    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549109    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549111    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549114    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549116    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549119    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549122    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549125    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:44:42.558802 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549127    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549130    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549132    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549135    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549138    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.549141    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.549706    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.557371    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.557385    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.558081    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.558088    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558153    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558158    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558163    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558167    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:44:42.559279 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558170    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558173    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558176    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558179    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558182    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558184    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558187    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558190    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558192    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558195    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558198    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558201    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558203    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558206    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558209    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558211    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558214    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558216    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558219    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558222    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:44:42.559663 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558224    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558227    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558229    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558232    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558234    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558238    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558241    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558244    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558247    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558250    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558252    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558255    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558258    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558260    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558263    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558265    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558268    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558270    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558273    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558276    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:44:42.560163 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558280    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558283    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558286    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558289    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558292    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558294    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558297    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558299    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558302    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558305    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558307    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558310    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558312    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558315    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558318    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558320    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558323    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558326    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558329    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558332    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:44:42.560679 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558335    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558338    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558341    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558343    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558346    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558349    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558351    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558354    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558356    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558359    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558361    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558364    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558367    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558369    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558372    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558375    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558377    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558380    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558382    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:44:42.561225 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558385    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558387    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558390    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558392    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558395    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558397    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558400    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558404    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558407    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558410    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558413    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558416    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558420    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558423    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558425    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:44:42.561716 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558428    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.558433    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558540    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558544    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558548    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558551    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558553    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558556    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558559    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558561    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558564    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558566    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558569    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558572    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558574    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 12:44:42.562161 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558577    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558579    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558582    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558584    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558588    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558591    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558594    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558596    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558599    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558602    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558604    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558607    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558610    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558613    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558616    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558620    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558623    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558626    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558629    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 12:44:42.562535 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558631    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558634    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558637    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558640    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558643    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558646    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558648    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558651    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558653    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558656    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558658    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558661    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558664    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558666    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558669    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558671    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558674    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558676    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558678    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558681    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 12:44:42.563005 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558683    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558686    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558688    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558691    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558693    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558696    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558701    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558704    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558707    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558710    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558713    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558715    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558718    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558720    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558723    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558726    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558728    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558731    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558733    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 12:44:42.563517 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558736    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558738    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558741    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558744    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558746    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558749    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558751    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558754    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558756    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558759    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558761    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558764    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558766    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558769    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558771    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558774    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558776    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558779    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558781    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558784    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 12:44:42.563987 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558787    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558789    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558792    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558795    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558797    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558800    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558803    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:42.558805    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.558810    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.558907    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.560767    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.561513    2562 server.go:1019] "Starting client certificate rotation"
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.561608    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 12:44:42.564474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.561637    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 12:44:42.564789 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.563948    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 12:44:42.568337 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.568315    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 12:44:42.579618 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.579582    2562 log.go:25] "Validated CRI v1 runtime API"
Sep 15 12:44:42.582444 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.582415    2562 log.go:25] "Validated CRI v1 image API"
Sep 15 12:44:42.583210 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.583195    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 12:44:42.585236 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.585217    2562 fs.go:135] Filesystem UUIDs: map[07aed628-38b5-44be-b7ab-130e462b1c55:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e73204e2-1cc3-4a48-b01c-6956f4d51dd3:/dev/nvme0n1p3]
Sep 15 12:44:42.585293 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.585235    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 12:44:42.590140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.590012    2562 manager.go:216] Machine: {Timestamp:2026-09-15 12:44:42.588850163 +0000 UTC m=+0.381551884 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3081850 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b7f6f5f495014259eb7bb132b3eaa SystemUUID:ec2b7f6f-5f49-5014-259e-b7bb132b3eaa BootID:b0ce9f15-926c-433d-b30b-6b4f7c8735fa Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e9:e3:63:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e9:e3:63:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:cd:fe:f2:83:56 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 12:44:42.590140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.590137    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 12:44:42.590244 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.590215    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 12:44:42.591229 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.591205    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 12:44:42.591353 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.591230    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-169.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 12:44:42.592430 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.592419    2562 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 12:44:42.592472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.592432    2562 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 12:44:42.592472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.592440    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 12:44:42.593433 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.593423    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 12:44:42.594188 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.594177    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 12:44:42.596118 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.596099    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 12:44:42.597085 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597074    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 12:44:42.597200 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597190    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 12:44:42.597270 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597220    2562 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 12:44:42.597270 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597232    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 12:44:42.597270 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597249    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 12:44:42.597270 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597258    2562 kubelet.go:398] "Adding apiserver pod source"
Sep 15 12:44:42.597270 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597267    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 12:44:42.597603 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597592    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 12:44:42.597644 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.597611    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 12:44:42.599556 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.599541    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 12:44:42.601394 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.601381    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 12:44:42.601435 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.601411    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 12:44:42.602930 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602917    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 12:44:42.602972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602935    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 12:44:42.602972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602942    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 12:44:42.602972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602947    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 12:44:42.602972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602956    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 12:44:42.602972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602962    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 12:44:42.603112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602975    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 12:44:42.603112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602980    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 12:44:42.603112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602987    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 12:44:42.603112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.602993    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 12:44:42.603112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.603009    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 12:44:42.603112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.603014    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 12:44:42.603689 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.603679    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 12:44:42.603689 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.603689    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 12:44:42.607656 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.607643    2562 server.go:1268] "Started kubelet"
Sep 15 12:44:42.607755 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.607725    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 12:44:42.607831 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.607789    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 12:44:42.607869 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.607850    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 12:44:42.609314 ip-10-0-132-169 systemd[1]: Started Kubernetes Kubelet.
Sep 15 12:44:42.610425 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.610408    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 12:44:42.610601 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.610587    2562 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 12:44:42.613432 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.613416    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 12:44:42.614394 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.614373    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 12:44:42.614394 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.614386    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 12:44:42.614541 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.614489    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:42.615090 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.615071    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 12:44:42.615090 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.615091    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 12:44:42.615232 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.615191    2562 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 12:44:42.615232 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.615199    2562 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 12:44:42.615232 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.615227    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 12:44:42.615871 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.615849    2562 factory.go:55] Registering systemd factory
Sep 15 12:44:42.615871 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.615868    2562 factory.go:223] Registration of the systemd container factory successfully
Sep 15 12:44:42.616396 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.616378    2562 factory.go:153] Registering CRI-O factory
Sep 15 12:44:42.616396 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.616397    2562 factory.go:223] Registration of the crio container factory successfully
Sep 15 12:44:42.616528 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.616448    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 12:44:42.616528 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.616464    2562 factory.go:103] Registering Raw factory
Sep 15 12:44:42.616528 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.616479    2562 manager.go:1196] Started watching for new ooms in manager
Sep 15 12:44:42.616942 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.616921    2562 manager.go:349] Starting recovery of all containers
Sep 15 12:44:42.617163 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.617140    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 12:44:42.621516 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.621488    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-169.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 12:44:42.621813 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.617239    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-169.ec2.internal.18d57f32aed03abe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-169.ec2.internal,UID:ip-10-0-132-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-169.ec2.internal,},FirstTimestamp:2026-09-15 12:44:42.607614654 +0000 UTC m=+0.400316377,LastTimestamp:2026-09-15 12:44:42.607614654 +0000 UTC m=+0.400316377,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-169.ec2.internal,}"
Sep 15 12:44:42.623969 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.623949    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-169.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 12:44:42.625528 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.625504    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 12:44:42.625763 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.625721    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-169.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 12:44:42.630893 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.630747    2562 manager.go:354] Recovery completed
Sep 15 12:44:42.636077 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.636059    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:44:42.638649 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.638629    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:44:42.638725 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.638662    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:44:42.638725 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.638677    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:44:42.639243 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.639229    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 12:44:42.639243 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.639242    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 12:44:42.639394 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.639259    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 12:44:42.639394 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.639295    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:44:42.640244 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.640229    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:44:42.640325 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.640260    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:44:42.640325 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.640275    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:44:42.640922 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.640907    2562 policy_none.go:49] "None policy: Start"
Sep 15 12:44:42.640963 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.640925    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 12:44:42.640963 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.640935    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 12:44:42.641818 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.641730    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-169.ec2.internal.18d57f32b0a9c6ff  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-169.ec2.internal,UID:ip-10-0-132-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-169.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-169.ec2.internal,},FirstTimestamp:2026-09-15 12:44:42.638649087 +0000 UTC m=+0.431350811,LastTimestamp:2026-09-15 12:44:42.638649087 +0000 UTC m=+0.431350811,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-169.ec2.internal,}"
Sep 15 12:44:42.641906 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.641826    2562 policy_none.go:47] "Start"
Sep 15 12:44:42.654752 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.654683    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-169.ec2.internal.18d57f32b0aa1502  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-169.ec2.internal,UID:ip-10-0-132-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-169.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-169.ec2.internal,},FirstTimestamp:2026-09-15 12:44:42.638669058 +0000 UTC m=+0.431370781,LastTimestamp:2026-09-15 12:44:42.638669058 +0000 UTC m=+0.431370781,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-169.ec2.internal,}"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.665321    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-169.ec2.internal.18d57f32b0aa4c2c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-169.ec2.internal,UID:ip-10-0-132-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-169.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-169.ec2.internal,},FirstTimestamp:2026-09-15 12:44:42.63868318 +0000 UTC m=+0.431384904,LastTimestamp:2026-09-15 12:44:42.63868318 +0000 UTC m=+0.431384904,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-169.ec2.internal,}"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.675564    2562 manager.go:337] "Starting Device Plugin manager"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.675600    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.675614    2562 server.go:87] "Starting device plugin registration server"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.675895    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.675906    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.675994    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.676085    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.676094    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.676579    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.676616    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.677129    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-169.ec2.internal.18d57f32b0a9c6ff\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-169.ec2.internal.18d57f32b0a9c6ff  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-169.ec2.internal,UID:ip-10-0-132-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-169.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-169.ec2.internal,},FirstTimestamp:2026-09-15 12:44:42.638649087 +0000 UTC m=+0.431350811,LastTimestamp:2026-09-15 12:44:42.640243992 +0000 UTC m=+0.432945743,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-169.ec2.internal,}"
Sep 15 12:44:42.684377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.677323    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qck66"
Sep 15 12:44:42.694555 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.694492    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-169.ec2.internal.18d57f32b0aa1502\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-169.ec2.internal.18d57f32b0aa1502  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-169.ec2.internal,UID:ip-10-0-132-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-169.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-169.ec2.internal,},FirstTimestamp:2026-09-15 12:44:42.638669058 +0000 UTC m=+0.431370781,LastTimestamp:2026-09-15 12:44:42.640266046 +0000 UTC m=+0.432967772,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-169.ec2.internal,}"
Sep 15 12:44:42.695065 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.695044    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qck66"
Sep 15 12:44:42.747915 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.747879    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 12:44:42.749050 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.749014    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 12:44:42.749137 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.749083    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 12:44:42.749137 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.749109    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 12:44:42.749237 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.749195    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 12:44:42.751511 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.751495    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:44:42.776445 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.776430    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:44:42.777252 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.777234    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:44:42.777326 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.777265    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:44:42.777326 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.777276    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:44:42.777326 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.777295    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-169.ec2.internal"
Sep 15 12:44:42.785995 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.785981    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-169.ec2.internal"
Sep 15 12:44:42.786052 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.786000    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-169.ec2.internal\": node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:42.825287 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.825228    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:42.850578 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.850071    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"]
Sep 15 12:44:42.850632 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.850616    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:44:42.851355 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.851339    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:44:42.851422 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.851366    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:44:42.851422 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.851376    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:44:42.852540 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.852528    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:44:42.852660 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.852646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:42.852698 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.852675    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:44:42.853211 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.853197    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:44:42.853273 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.853223    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:44:42.853273 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.853230    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:44:42.853273 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.853252    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:44:42.853273 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.853263    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:44:42.853406 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.853236    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:44:42.854337 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.854324    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:42.854388 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.854346    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:44:42.854949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.854936    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:44:42.855010 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.854962    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:44:42.855010 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:42.854976    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:44:42.879616 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.879599    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-169.ec2.internal\" not found" node="ip-10-0-132-169.ec2.internal"
Sep 15 12:44:42.883867 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.883849    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-169.ec2.internal\" not found" node="ip-10-0-132-169.ec2.internal"
Sep 15 12:44:42.926083 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:42.926067    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.016433 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.016412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40993ab6a5e5658e373011e0ad6bc90a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-169.ec2.internal\" (UID: \"40993ab6a5e5658e373011e0ad6bc90a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.016544 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.016441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e3093d9551f21047f854f90d1e65ac15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"e3093d9551f21047f854f90d1e65ac15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.016544 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.016462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3093d9551f21047f854f90d1e65ac15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"e3093d9551f21047f854f90d1e65ac15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.026523 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.026501    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.117222 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.117168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40993ab6a5e5658e373011e0ad6bc90a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-169.ec2.internal\" (UID: \"40993ab6a5e5658e373011e0ad6bc90a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.117222 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.117197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e3093d9551f21047f854f90d1e65ac15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"e3093d9551f21047f854f90d1e65ac15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.117222 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.117213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3093d9551f21047f854f90d1e65ac15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"e3093d9551f21047f854f90d1e65ac15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.117355 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.117282    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3093d9551f21047f854f90d1e65ac15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"e3093d9551f21047f854f90d1e65ac15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.117355 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.117288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e3093d9551f21047f854f90d1e65ac15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"e3093d9551f21047f854f90d1e65ac15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.117355 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.117323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40993ab6a5e5658e373011e0ad6bc90a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-169.ec2.internal\" (UID: \"40993ab6a5e5658e373011e0ad6bc90a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.127292 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.127271    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.182493 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.182456    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.188291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.188274    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:43.227435 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.227409    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.328474 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.328444    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.429523 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.429462    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.529630 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.529599    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.561765 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.561732    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 12:44:43.562197 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.561874    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 12:44:43.630361 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.630336    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.634909 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.634889    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:44:43.691838 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.691777    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 12:44:43.693360 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.693340    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 12:44:43.693523 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.693376    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 12:44:43.693523 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.693391    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID"
Sep 15 12:44:43.701560 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.701533    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 12:39:42 +0000 UTC" deadline="2028-03-24 21:08:41.512664159 +0000 UTC"
Sep 15 12:44:43.701560 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.701559    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13352h23m57.811108319s"
Sep 15 12:44:43.731250 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.731224    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.798121 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:43.798057    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40993ab6a5e5658e373011e0ad6bc90a.slice/crio-7b2a88384c70298eec256dae35557d6459c65b8ec8b5cd1cf26c17d277e87fe4 WatchSource:0}: Error finding container 7b2a88384c70298eec256dae35557d6459c65b8ec8b5cd1cf26c17d277e87fe4: Status 404 returned error can't find the container with id 7b2a88384c70298eec256dae35557d6459c65b8ec8b5cd1cf26c17d277e87fe4
Sep 15 12:44:43.798382 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:43.798360    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3093d9551f21047f854f90d1e65ac15.slice/crio-6d7f9a055308fa149449b2058306b9e1b1e77938f67fe320bf3564d21703aa16 WatchSource:0}: Error finding container 6d7f9a055308fa149449b2058306b9e1b1e77938f67fe320bf3564d21703aa16: Status 404 returned error can't find the container with id 6d7f9a055308fa149449b2058306b9e1b1e77938f67fe320bf3564d21703aa16
Sep 15 12:44:43.803289 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:43.803274    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:44:43.831643 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.831622    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:43.931776 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:43.931741    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:44.017531 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.017467    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:44:44.032764 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.032743    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:44.133902 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.133877    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found"
Sep 15 12:44:44.175848 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.175822    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:44:44.217707 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.217680    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:44.234854 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.234833    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 12:44:44.235855 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.235836    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"
Sep 15 12:44:44.243122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.243105    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 12:44:44.367767 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.367694    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 12:44:44.599201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.599172    2562 apiserver.go:52] "Watching apiserver"
Sep 15 12:44:44.611132 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.611102    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 12:44:44.611538 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.611505    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-qk4m8","openshift-dns/node-resolver-nb6bf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal","openshift-multus/multus-vb7k8","openshift-network-operator/iptables-alerter-bl8tb","kube-system/konnectivity-agent-lbdf9","kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal","openshift-image-registry/node-ca-vq2mc","openshift-multus/multus-additional-cni-plugins-vlghj","openshift-multus/network-metrics-daemon-wdsdq","openshift-network-diagnostics/network-check-target-sm5v4","openshift-ovn-kubernetes/ovnkube-node-7qpxx","kube-system/global-pull-secret-syncer-9wmfs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"]
Sep 15 12:44:44.612975 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.612955    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:44.613122 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.613065    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:44:44.614168 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.614147    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.614269 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.614220    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:44:44.615350 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.615320    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.615642 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.615599    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 12:44:44.616594 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.616574    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.618029 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.617760    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.618029 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.617894    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ht57z\""
Sep 15 12:44:44.618761 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.618654    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 12:44:44.618761 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.618677    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 12:44:44.619228 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.619202    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 12:44:44.619524 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.619504    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:44:44.621190 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.621164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.623266 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.623246    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 12:44:44.623628 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.623356    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 12:44:44.623770 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.623624    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 12:44:44.625272 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.624863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.625272 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.624899    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 12:44:44.625272 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625142    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 12:44:44.625492 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625301    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 12:44:44.625492 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625398    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:44:44.625492 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625469    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 12:44:44.625648 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625502    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-j58xz\""
Sep 15 12:44:44.625648 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625537    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 12:44:44.625648 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625603    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qxfgb\""
Sep 15 12:44:44.625648 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625611    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7q469\""
Sep 15 12:44:44.625882 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625760    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 12:44:44.625882 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625815    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 12:44:44.625882 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.625816    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xhg62\""
Sep 15 12:44:44.626091 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.626076    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.627790 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.627767    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 12:44:44.628853 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.628836    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.630042 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.629930    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-cnibin\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630042 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.629953    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s2qj4\""
Sep 15 12:44:44.630042 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.629971    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d773ad9-c704-4c02-bac5-855192af726f-cni-binary-copy\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630042 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630005    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffa614d2-e094-43ee-a793-042735c1161d-kubelet-config\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.630042 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-conf-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-multus-certs\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-socket-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630158    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-device-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630193    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2krd\" (UniqueName: \"kubernetes.io/projected/b089d305-66c1-45c1-8559-9cdc2632cc81-kube-api-access-b2krd\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630217    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-system-cni-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630288    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:44.630327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-cni-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630355    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-cni-multus\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.630358    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-registration-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630427    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-sys-fs\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-etc-kubernetes\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-etc-selinux\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630516    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-cni-bin\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d773ad9-c704-4c02-bac5-855192af726f-multus-daemon-config\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-k8s-cni-cncf-io\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630608    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n74zf\" (UniqueName: \"kubernetes.io/projected/1d773ad9-c704-4c02-bac5-855192af726f-kube-api-access-n74zf\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c4j7\" (UniqueName: \"kubernetes.io/projected/a39d331a-f507-4475-9a80-16a20f6339b6-kube-api-access-6c4j7\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffa614d2-e094-43ee-a793-042735c1161d-dbus\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.630667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-netns\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.631141 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-kubelet\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.631141 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-hostroot\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.631141 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630792    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-socket-dir-parent\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.631141 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:44.631141 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.630846    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.631411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.631396    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.631642 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.631616    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 12:44:44.631736 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.631654    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 12:44:44.631799 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.631773    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5wxm2\""
Sep 15 12:44:44.631799 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.631784    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:44:44.632049 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.632014    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-svzkg\""
Sep 15 12:44:44.632571 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.632557    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 12:44:44.639338 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639171    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 12:44:44.639338 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639209    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 12:44:44.639338 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639235    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lwv27\""
Sep 15 12:44:44.639555 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639351    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 12:44:44.639555 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639433    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 12:44:44.639555 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639519    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 12:44:44.639555 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639550    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 12:44:44.639722 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639613    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 12:44:44.639825 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.639812    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 12:44:44.667041 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.667003    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nfmv6"
Sep 15 12:44:44.682453 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.682390    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nfmv6"
Sep 15 12:44:44.716765 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.716740    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 12:44:44.731782 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731755    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eedbe00f-27fc-4124-b091-c64793e031d3-agent-certs\") pod \"konnectivity-agent-lbdf9\" (UID: \"eedbe00f-27fc-4124-b091-c64793e031d3\") " pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:44:44.731912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-cni-bin\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.731912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-modprobe-d\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.731912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731824    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34069473-d7cd-4964-b84a-ef8caa635898-tmp\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.731912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-system-cni-dir\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.731912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731886    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.731912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731894    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-cni-bin\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.731912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731913    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-k8s-cni-cncf-io\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n74zf\" (UniqueName: \"kubernetes.io/projected/1d773ad9-c704-4c02-bac5-855192af726f-kube-api-access-n74zf\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731950    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-k8s-cni-cncf-io\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731969    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a544b24-4e29-4395-84fe-3d2e1086f6bd-hosts-file\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.731986    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-kubernetes\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-etc-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732025    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-cni-netd\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732048    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c4j7\" (UniqueName: \"kubernetes.io/projected/a39d331a-f507-4475-9a80-16a20f6339b6-kube-api-access-6c4j7\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-hostroot\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysctl-conf\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732139    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-hostroot\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-node-log\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-socket-dir-parent\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-run\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-systemd\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-socket-dir-parent\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.732351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-cnibin\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.732414    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732432    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d773ad9-c704-4c02-bac5-855192af726f-cni-binary-copy\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.732468    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs podName:a39d331a-f507-4475-9a80-16a20f6339b6 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:45.232451011 +0000 UTC m=+3.025152721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs") pod "network-metrics-daemon-wdsdq" (UID: "a39d331a-f507-4475-9a80-16a20f6339b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovnkube-config\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-cnibin\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732596    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffa614d2-e094-43ee-a793-042735c1161d-kubelet-config\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-multus-certs\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffa614d2-e094-43ee-a793-042735c1161d-kubelet-config\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c464e89d-901f-42bf-a59d-04f381376587-iptables-alerter-script\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysconfig\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732679    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-multus-certs\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732710    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnds4\" (UniqueName: \"kubernetes.io/projected/2344c353-bb2a-496e-b049-844dd8edf31c-kube-api-access-bnds4\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-kubelet\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh7tz\" (UniqueName: \"kubernetes.io/projected/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-kube-api-access-kh7tz\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-socket-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-system-cni-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733214 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-cni-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-cni-multus\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-registration-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-sys-fs\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-socket-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c464e89d-901f-42bf-a59d-04f381376587-host-slash\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732976    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysctl-d\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732985    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-system-cni-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733024    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-cni-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733029    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733026    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d773ad9-c704-4c02-bac5-855192af726f-cni-binary-copy\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.732976    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-cni-multus\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-env-overrides\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-registration-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-etc-selinux\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733134    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-sys-fs\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.733943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733142    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d773ad9-c704-4c02-bac5-855192af726f-multus-daemon-config\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-host\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733178    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-etc-selinux\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-run-ovn-kubernetes\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733254    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-cni-bin\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8gkh\" (UniqueName: \"kubernetes.io/projected/c3351e98-4182-4ebd-bccd-594fbb589096-kube-api-access-q8gkh\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sl7n\" (UniqueName: \"kubernetes.io/projected/3a544b24-4e29-4395-84fe-3d2e1086f6bd-kube-api-access-8sl7n\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733345    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eedbe00f-27fc-4124-b091-c64793e031d3-konnectivity-ca\") pod \"konnectivity-agent-lbdf9\" (UID: \"eedbe00f-27fc-4124-b091-c64793e031d3\") " pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-cnibin\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffa614d2-e094-43ee-a793-042735c1161d-dbus\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-netns\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-kubelet\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-run-netns\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-systemd\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733517    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-host-var-lib-kubelet\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.734574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733525    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-sys\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffa614d2-e094-43ee-a793-042735c1161d-dbus\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733553    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a544b24-4e29-4395-84fe-3d2e1086f6bd-tmp-dir\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-var-lib-kubelet\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733576    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d773ad9-c704-4c02-bac5-855192af726f-multus-daemon-config\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733590    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-run-netns\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-var-lib-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733732    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-conf-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.733777    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3351e98-4182-4ebd-bccd-594fbb589096-host\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733786    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-lib-modules\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.733829    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret podName:ffa614d2-e094-43ee-a793-042735c1161d nodeName:}" failed. No retries permitted until 2026-09-15 12:44:45.233813339 +0000 UTC m=+3.026515050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret") pod "global-pull-secret-syncer-9wmfs" (UID: "ffa614d2-e094-43ee-a793-042735c1161d") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-multus-conf-dir\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.735138 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733863    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qmtl\" (UniqueName: \"kubernetes.io/projected/34069473-d7cd-4964-b84a-ef8caa635898-kube-api-access-4qmtl\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-ovn\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-log-socket\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733927    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-device-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2krd\" (UniqueName: \"kubernetes.io/projected/b089d305-66c1-45c1-8559-9cdc2632cc81-kube-api-access-b2krd\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733957    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2krbp\" (UniqueName: \"kubernetes.io/projected/c464e89d-901f-42bf-a59d-04f381376587-kube-api-access-2krbp\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3351e98-4182-4ebd-bccd-594fbb589096-serviceca\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.733994    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/34069473-d7cd-4964-b84a-ef8caa635898-etc-tuned\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.734008    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-systemd-units\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.734054    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-slash\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.734075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.734144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-etc-kubernetes\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.734164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovn-node-metrics-cert\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.734180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovnkube-script-lib\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.734220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d773ad9-c704-4c02-bac5-855192af726f-etc-kubernetes\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.735663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.734237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b089d305-66c1-45c1-8559-9cdc2632cc81-device-dir\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.744159 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.744128    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 12:44:44.748067 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.748041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2krd\" (UniqueName: \"kubernetes.io/projected/b089d305-66c1-45c1-8559-9cdc2632cc81-kube-api-access-b2krd\") pod \"aws-ebs-csi-driver-node-zwskb\" (UID: \"b089d305-66c1-45c1-8559-9cdc2632cc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.748067 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.748060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n74zf\" (UniqueName: \"kubernetes.io/projected/1d773ad9-c704-4c02-bac5-855192af726f-kube-api-access-n74zf\") pod \"multus-vb7k8\" (UID: \"1d773ad9-c704-4c02-bac5-855192af726f\") " pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.748400 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.748382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c4j7\" (UniqueName: \"kubernetes.io/projected/a39d331a-f507-4475-9a80-16a20f6339b6-kube-api-access-6c4j7\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:44.754902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.754862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" event={"ID":"40993ab6a5e5658e373011e0ad6bc90a","Type":"ContainerStarted","Data":"7b2a88384c70298eec256dae35557d6459c65b8ec8b5cd1cf26c17d277e87fe4"}
Sep 15 12:44:44.756240 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.756210    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" event={"ID":"e3093d9551f21047f854f90d1e65ac15","Type":"ContainerStarted","Data":"6d7f9a055308fa149449b2058306b9e1b1e77938f67fe320bf3564d21703aa16"}
Sep 15 12:44:44.834872 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.834841    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a544b24-4e29-4395-84fe-3d2e1086f6bd-hosts-file\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.834872 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.834875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-kubernetes\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.834894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-etc-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-cni-netd\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a544b24-4e29-4395-84fe-3d2e1086f6bd-hosts-file\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835076    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysctl-conf\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835083    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-cni-netd\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-etc-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-node-log\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835037    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-kubernetes\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-run\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-systemd\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835171    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-node-log\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-run\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysctl-conf\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovnkube-config\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835252    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-systemd\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835267    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c464e89d-901f-42bf-a59d-04f381376587-iptables-alerter-script\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835295    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysconfig\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnds4\" (UniqueName: \"kubernetes.io/projected/2344c353-bb2a-496e-b049-844dd8edf31c-kube-api-access-bnds4\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-kubelet\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835379    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh7tz\" (UniqueName: \"kubernetes.io/projected/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-kube-api-access-kh7tz\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-kubelet\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c464e89d-901f-42bf-a59d-04f381376587-host-slash\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysctl-d\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835356    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysconfig\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.835607 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-env-overrides\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c464e89d-901f-42bf-a59d-04f381376587-host-slash\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835648    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-host\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835732    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-run-ovn-kubernetes\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-cni-bin\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8gkh\" (UniqueName: \"kubernetes.io/projected/c3351e98-4182-4ebd-bccd-594fbb589096-kube-api-access-q8gkh\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835816    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sl7n\" (UniqueName: \"kubernetes.io/projected/3a544b24-4e29-4395-84fe-3d2e1086f6bd-kube-api-access-8sl7n\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835815    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eedbe00f-27fc-4124-b091-c64793e031d3-konnectivity-ca\") pod \"konnectivity-agent-lbdf9\" (UID: \"eedbe00f-27fc-4124-b091-c64793e031d3\") " pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c464e89d-901f-42bf-a59d-04f381376587-iptables-alerter-script\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835888    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovnkube-config\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-cnibin\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-cnibin\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-systemd\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.836486 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-sys\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.835992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a544b24-4e29-4395-84fe-3d2e1086f6bd-tmp-dir\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-var-lib-kubelet\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-run-ovn-kubernetes\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-systemd\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836157    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-var-lib-kubelet\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836184    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-sys\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836190    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-cni-bin\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-sysctl-d\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-host\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-run-netns\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-var-lib-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-run-netns\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836422    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eedbe00f-27fc-4124-b091-c64793e031d3-konnectivity-ca\") pod \"konnectivity-agent-lbdf9\" (UID: \"eedbe00f-27fc-4124-b091-c64793e031d3\") " pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836392    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3351e98-4182-4ebd-bccd-594fbb589096-host\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.837423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3351e98-4182-4ebd-bccd-594fbb589096-host\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836492    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a544b24-4e29-4395-84fe-3d2e1086f6bd-tmp-dir\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-lib-modules\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qmtl\" (UniqueName: \"kubernetes.io/projected/34069473-d7cd-4964-b84a-ef8caa635898-kube-api-access-4qmtl\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-lib-modules\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-var-lib-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-ovn\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-ovn\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-log-socket\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836659    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-env-overrides\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836674    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2krbp\" (UniqueName: \"kubernetes.io/projected/c464e89d-901f-42bf-a59d-04f381376587-kube-api-access-2krbp\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836687    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-log-socket\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3351e98-4182-4ebd-bccd-594fbb589096-serviceca\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/34069473-d7cd-4964-b84a-ef8caa635898-etc-tuned\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-systemd-units\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2344c353-bb2a-496e-b049-844dd8edf31c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-systemd-units\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.838483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836922    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-slash\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.836999    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovn-node-metrics-cert\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-slash\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovnkube-script-lib\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3351e98-4182-4ebd-bccd-594fbb589096-serviceca\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eedbe00f-27fc-4124-b091-c64793e031d3-agent-certs\") pod \"konnectivity-agent-lbdf9\" (UID: \"eedbe00f-27fc-4124-b091-c64793e031d3\") " pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-modprobe-d\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34069473-d7cd-4964-b84a-ef8caa635898-tmp\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-system-cni-dir\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837286    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/34069473-d7cd-4964-b84a-ef8caa635898-etc-modprobe-d\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837364    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2344c353-bb2a-496e-b049-844dd8edf31c-system-cni-dir\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837364    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-run-openvswitch\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.839403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.837562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovnkube-script-lib\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.840096 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.839945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-ovn-node-metrics-cert\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.840096 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.839943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/34069473-d7cd-4964-b84a-ef8caa635898-etc-tuned\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.840334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.840311    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34069473-d7cd-4964-b84a-ef8caa635898-tmp\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.840427 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.840410    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eedbe00f-27fc-4124-b091-c64793e031d3-agent-certs\") pod \"konnectivity-agent-lbdf9\" (UID: \"eedbe00f-27fc-4124-b091-c64793e031d3\") " pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:44:44.845991 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.845968    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:44:44.846116 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.845997    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:44:44.846116 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.846012    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bh8d5 for pod openshift-network-diagnostics/network-check-target-sm5v4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:44.846116 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:44.846084    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5 podName:86965e08-cf54-4abf-a78a-b377d4e2d931 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:45.34606659 +0000 UTC m=+3.138768310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bh8d5" (UniqueName: "kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5") pod "network-check-target-sm5v4" (UID: "86965e08-cf54-4abf-a78a-b377d4e2d931") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:44.848851 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.848777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sl7n\" (UniqueName: \"kubernetes.io/projected/3a544b24-4e29-4395-84fe-3d2e1086f6bd-kube-api-access-8sl7n\") pod \"node-resolver-nb6bf\" (UID: \"3a544b24-4e29-4395-84fe-3d2e1086f6bd\") " pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.852142 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.852098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qmtl\" (UniqueName: \"kubernetes.io/projected/34069473-d7cd-4964-b84a-ef8caa635898-kube-api-access-4qmtl\") pod \"tuned-qk4m8\" (UID: \"34069473-d7cd-4964-b84a-ef8caa635898\") " pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.852362 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.852337    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh7tz\" (UniqueName: \"kubernetes.io/projected/b6a65b2e-c744-46f4-b9ca-b7166c2f9107-kube-api-access-kh7tz\") pod \"ovnkube-node-7qpxx\" (UID: \"b6a65b2e-c744-46f4-b9ca-b7166c2f9107\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:44.852513 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.852488    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnds4\" (UniqueName: \"kubernetes.io/projected/2344c353-bb2a-496e-b049-844dd8edf31c-kube-api-access-bnds4\") pod \"multus-additional-cni-plugins-vlghj\" (UID: \"2344c353-bb2a-496e-b049-844dd8edf31c\") " pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.852588 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.852505    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2krbp\" (UniqueName: \"kubernetes.io/projected/c464e89d-901f-42bf-a59d-04f381376587-kube-api-access-2krbp\") pod \"iptables-alerter-bl8tb\" (UID: \"c464e89d-901f-42bf-a59d-04f381376587\") " pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.853534 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.853511    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8gkh\" (UniqueName: \"kubernetes.io/projected/c3351e98-4182-4ebd-bccd-594fbb589096-kube-api-access-q8gkh\") pod \"node-ca-vq2mc\" (UID: \"c3351e98-4182-4ebd-bccd-594fbb589096\") " pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.928687 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.928602    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb"
Sep 15 12:44:44.936535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.936513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vb7k8"
Sep 15 12:44:44.944747 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.944726    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bl8tb"
Sep 15 12:44:44.948397 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.948375    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:44:44.955045 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.955022    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vlghj"
Sep 15 12:44:44.960673 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.960653    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vq2mc"
Sep 15 12:44:44.968188 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.968169    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nb6bf"
Sep 15 12:44:44.974965 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.974947    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qk4m8"
Sep 15 12:44:44.988632 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:44.988607    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:44:45.074210 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.074183    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2344c353_bb2a_496e_b049_844dd8edf31c.slice/crio-202391b8f0d3fdf223b80273ea8bddf9b2bf20b6b1ea158c816508deb03d3dc6 WatchSource:0}: Error finding container 202391b8f0d3fdf223b80273ea8bddf9b2bf20b6b1ea158c816508deb03d3dc6: Status 404 returned error can't find the container with id 202391b8f0d3fdf223b80273ea8bddf9b2bf20b6b1ea158c816508deb03d3dc6
Sep 15 12:44:45.075275 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.075135    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc464e89d_901f_42bf_a59d_04f381376587.slice/crio-8770f46ec1d84f2f4fa7f17dffb166d4b12ad73ad0f0dcf3486a127faa6833a2 WatchSource:0}: Error finding container 8770f46ec1d84f2f4fa7f17dffb166d4b12ad73ad0f0dcf3486a127faa6833a2: Status 404 returned error can't find the container with id 8770f46ec1d84f2f4fa7f17dffb166d4b12ad73ad0f0dcf3486a127faa6833a2
Sep 15 12:44:45.076763 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.076738    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb089d305_66c1_45c1_8559_9cdc2632cc81.slice/crio-fab234cf9d25783928311b831252e0b8cd276237830cc6aa2b7651ae6a9c3c7b WatchSource:0}: Error finding container fab234cf9d25783928311b831252e0b8cd276237830cc6aa2b7651ae6a9c3c7b: Status 404 returned error can't find the container with id fab234cf9d25783928311b831252e0b8cd276237830cc6aa2b7651ae6a9c3c7b
Sep 15 12:44:45.081490 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.081465    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d773ad9_c704_4c02_bac5_855192af726f.slice/crio-cdceeaf6d4321327f3274349e3b3246385915ccd3b759c7b82065227ea1822a0 WatchSource:0}: Error finding container cdceeaf6d4321327f3274349e3b3246385915ccd3b759c7b82065227ea1822a0: Status 404 returned error can't find the container with id cdceeaf6d4321327f3274349e3b3246385915ccd3b759c7b82065227ea1822a0
Sep 15 12:44:45.083200 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.083175    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a65b2e_c744_46f4_b9ca_b7166c2f9107.slice/crio-96f709a70b127871da07dfa1b2fa317362e3870599aaa4707b6ff5cd09cf9f09 WatchSource:0}: Error finding container 96f709a70b127871da07dfa1b2fa317362e3870599aaa4707b6ff5cd09cf9f09: Status 404 returned error can't find the container with id 96f709a70b127871da07dfa1b2fa317362e3870599aaa4707b6ff5cd09cf9f09
Sep 15 12:44:45.084579 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.084556    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3351e98_4182_4ebd_bccd_594fbb589096.slice/crio-0f597def974007603a91730ea782ddf480a4b6d07e7431f85af56b5d15a3bb49 WatchSource:0}: Error finding container 0f597def974007603a91730ea782ddf480a4b6d07e7431f85af56b5d15a3bb49: Status 404 returned error can't find the container with id 0f597def974007603a91730ea782ddf480a4b6d07e7431f85af56b5d15a3bb49
Sep 15 12:44:45.086826 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.086690    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a544b24_4e29_4395_84fe_3d2e1086f6bd.slice/crio-64fb354a6be07f34bc33dc1cb24fcbb8b234188b996c0b79ca05fcf1d29dee7d WatchSource:0}: Error finding container 64fb354a6be07f34bc33dc1cb24fcbb8b234188b996c0b79ca05fcf1d29dee7d: Status 404 returned error can't find the container with id 64fb354a6be07f34bc33dc1cb24fcbb8b234188b996c0b79ca05fcf1d29dee7d
Sep 15 12:44:45.088421 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.088344    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeedbe00f_27fc_4124_b091_c64793e031d3.slice/crio-8ac26ba31d632babe92b5dfa438326c93d77f23ea373bbe28f3d73024608c3ba WatchSource:0}: Error finding container 8ac26ba31d632babe92b5dfa438326c93d77f23ea373bbe28f3d73024608c3ba: Status 404 returned error can't find the container with id 8ac26ba31d632babe92b5dfa438326c93d77f23ea373bbe28f3d73024608c3ba
Sep 15 12:44:45.089249 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:44:45.089230    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34069473_d7cd_4964_b84a_ef8caa635898.slice/crio-5be4966f00e5c4ea190b8ed3cad895a2a9583dee1e14a32284e32e38604be547 WatchSource:0}: Error finding container 5be4966f00e5c4ea190b8ed3cad895a2a9583dee1e14a32284e32e38604be547: Status 404 returned error can't find the container with id 5be4966f00e5c4ea190b8ed3cad895a2a9583dee1e14a32284e32e38604be547
Sep 15 12:44:45.240972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.240737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:45.240972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.240951    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:45.241232 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:45.240894    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:45.241232 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:45.241052    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:45.241232 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:45.241088    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs podName:a39d331a-f507-4475-9a80-16a20f6339b6 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:46.241059736 +0000 UTC m=+4.033761464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs") pod "network-metrics-daemon-wdsdq" (UID: "a39d331a-f507-4475-9a80-16a20f6339b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:45.241232 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:45.241108    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret podName:ffa614d2-e094-43ee-a793-042735c1161d nodeName:}" failed. No retries permitted until 2026-09-15 12:44:46.241097017 +0000 UTC m=+4.033798728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret") pod "global-pull-secret-syncer-9wmfs" (UID: "ffa614d2-e094-43ee-a793-042735c1161d") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:45.442170 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.442130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:45.442353 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:45.442330    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:44:45.442459 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:45.442441    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:44:45.442459 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:45.442458    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bh8d5 for pod openshift-network-diagnostics/network-check-target-sm5v4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:45.442611 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:45.442531    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5 podName:86965e08-cf54-4abf-a78a-b377d4e2d931 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:46.44250624 +0000 UTC m=+4.235207953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bh8d5" (UniqueName: "kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5") pod "network-check-target-sm5v4" (UID: "86965e08-cf54-4abf-a78a-b377d4e2d931") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:45.685815 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.685666    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 12:39:44 +0000 UTC" deadline="2028-02-09 22:48:28.639131222 +0000 UTC"
Sep 15 12:44:45.685815 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.685710    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12298h3m42.953429954s"
Sep 15 12:44:45.761230 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.761085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qk4m8" event={"ID":"34069473-d7cd-4964-b84a-ef8caa635898","Type":"ContainerStarted","Data":"5be4966f00e5c4ea190b8ed3cad895a2a9583dee1e14a32284e32e38604be547"}
Sep 15 12:44:45.763359 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.763325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lbdf9" event={"ID":"eedbe00f-27fc-4124-b091-c64793e031d3","Type":"ContainerStarted","Data":"8ac26ba31d632babe92b5dfa438326c93d77f23ea373bbe28f3d73024608c3ba"}
Sep 15 12:44:45.765175 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.765081    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nb6bf" event={"ID":"3a544b24-4e29-4395-84fe-3d2e1086f6bd","Type":"ContainerStarted","Data":"64fb354a6be07f34bc33dc1cb24fcbb8b234188b996c0b79ca05fcf1d29dee7d"}
Sep 15 12:44:45.768264 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.768168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vq2mc" event={"ID":"c3351e98-4182-4ebd-bccd-594fbb589096","Type":"ContainerStarted","Data":"0f597def974007603a91730ea782ddf480a4b6d07e7431f85af56b5d15a3bb49"}
Sep 15 12:44:45.772352 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.772226    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"96f709a70b127871da07dfa1b2fa317362e3870599aaa4707b6ff5cd09cf9f09"}
Sep 15 12:44:45.775264 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.775236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vb7k8" event={"ID":"1d773ad9-c704-4c02-bac5-855192af726f","Type":"ContainerStarted","Data":"cdceeaf6d4321327f3274349e3b3246385915ccd3b759c7b82065227ea1822a0"}
Sep 15 12:44:45.777567 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.777545    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb" event={"ID":"b089d305-66c1-45c1-8559-9cdc2632cc81","Type":"ContainerStarted","Data":"fab234cf9d25783928311b831252e0b8cd276237830cc6aa2b7651ae6a9c3c7b"}
Sep 15 12:44:45.782973 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.782943    2562 generic.go:356] "Generic (PLEG): container finished" podID="e3093d9551f21047f854f90d1e65ac15" containerID="12c3ffa5c2c392366e9aee05ca53104465b3a864ece21caa37c4b6e004af571c" exitCode=0
Sep 15 12:44:45.783328 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.783248    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" event={"ID":"e3093d9551f21047f854f90d1e65ac15","Type":"ContainerDied","Data":"12c3ffa5c2c392366e9aee05ca53104465b3a864ece21caa37c4b6e004af571c"}
Sep 15 12:44:45.786185 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.786161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bl8tb" event={"ID":"c464e89d-901f-42bf-a59d-04f381376587","Type":"ContainerStarted","Data":"8770f46ec1d84f2f4fa7f17dffb166d4b12ad73ad0f0dcf3486a127faa6833a2"}
Sep 15 12:44:45.794529 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:45.794497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlghj" event={"ID":"2344c353-bb2a-496e-b049-844dd8edf31c","Type":"ContainerStarted","Data":"202391b8f0d3fdf223b80273ea8bddf9b2bf20b6b1ea158c816508deb03d3dc6"}
Sep 15 12:44:46.249511 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.249414    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:46.249688 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.249518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:46.249688 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.249670    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:46.249801 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.249737    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs podName:a39d331a-f507-4475-9a80-16a20f6339b6 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:48.249716813 +0000 UTC m=+6.042418528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs") pod "network-metrics-daemon-wdsdq" (UID: "a39d331a-f507-4475-9a80-16a20f6339b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:46.250191 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.250172    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:46.250308 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.250231    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret podName:ffa614d2-e094-43ee-a793-042735c1161d nodeName:}" failed. No retries permitted until 2026-09-15 12:44:48.250213747 +0000 UTC m=+6.042915463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret") pod "global-pull-secret-syncer-9wmfs" (UID: "ffa614d2-e094-43ee-a793-042735c1161d") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:46.451727 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.451686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:46.451921 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.451864    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:44:46.451921 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.451885    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:44:46.451921 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.451900    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bh8d5 for pod openshift-network-diagnostics/network-check-target-sm5v4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:46.452107 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.451969    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5 podName:86965e08-cf54-4abf-a78a-b377d4e2d931 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:48.45194924 +0000 UTC m=+6.244650967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bh8d5" (UniqueName: "kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5") pod "network-check-target-sm5v4" (UID: "86965e08-cf54-4abf-a78a-b377d4e2d931") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:46.686955 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.686838    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 12:39:44 +0000 UTC" deadline="2028-05-26 06:19:41.042667944 +0000 UTC"
Sep 15 12:44:46.686955 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.686902    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14849h34m54.355770819s"
Sep 15 12:44:46.751256 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.751226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:46.751432 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.751358    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:44:46.751802 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.751779    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:46.751901 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.751884    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:44:46.753824 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.753668    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:46.753824 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:46.753780    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:44:46.806216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:46.806182    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" event={"ID":"e3093d9551f21047f854f90d1e65ac15","Type":"ContainerStarted","Data":"d09e132f2120c2f029fc90f6cfb9563c5aef41eb86b0c9b52cef96c468cfa26b"}
Sep 15 12:44:48.267216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:48.267166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:48.267820 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:48.267257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:48.267820 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.267406    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:48.267820 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.267466    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs podName:a39d331a-f507-4475-9a80-16a20f6339b6 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:52.267447867 +0000 UTC m=+10.060149593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs") pod "network-metrics-daemon-wdsdq" (UID: "a39d331a-f507-4475-9a80-16a20f6339b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:48.268151 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.268126    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:48.268241 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.268187    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret podName:ffa614d2-e094-43ee-a793-042735c1161d nodeName:}" failed. No retries permitted until 2026-09-15 12:44:52.26816794 +0000 UTC m=+10.060869662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret") pod "global-pull-secret-syncer-9wmfs" (UID: "ffa614d2-e094-43ee-a793-042735c1161d") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:48.468383 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:48.468343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:48.468621 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.468511    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:44:48.468621 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.468532    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:44:48.468621 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.468545    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bh8d5 for pod openshift-network-diagnostics/network-check-target-sm5v4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:48.468621 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.468615    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5 podName:86965e08-cf54-4abf-a78a-b377d4e2d931 nodeName:}" failed. No retries permitted until 2026-09-15 12:44:52.468583834 +0000 UTC m=+10.261285565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bh8d5" (UniqueName: "kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5") pod "network-check-target-sm5v4" (UID: "86965e08-cf54-4abf-a78a-b377d4e2d931") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:48.750251 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:48.750118    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:48.750251 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:48.750135    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:48.750449 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.750259    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:44:48.750449 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.750347    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:44:48.750449 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:48.750410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:48.750609 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:48.750484    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:44:50.749925 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:50.749769    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:50.750395 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:50.749926    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:44:50.750395 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:50.749936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:50.750395 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:50.749970    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:50.750395 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:50.750019    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:44:50.750395 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:50.750108    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:44:52.300084 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:52.300044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:52.300578 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:52.300122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:52.300578 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.300308    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:52.300578 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.300381    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret podName:ffa614d2-e094-43ee-a793-042735c1161d nodeName:}" failed. No retries permitted until 2026-09-15 12:45:00.300360146 +0000 UTC m=+18.093061861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret") pod "global-pull-secret-syncer-9wmfs" (UID: "ffa614d2-e094-43ee-a793-042735c1161d") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:44:52.300578 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.300386    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:52.300578 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.300453    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs podName:a39d331a-f507-4475-9a80-16a20f6339b6 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:00.300435349 +0000 UTC m=+18.093137077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs") pod "network-metrics-daemon-wdsdq" (UID: "a39d331a-f507-4475-9a80-16a20f6339b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:44:52.501749 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:52.501707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:52.501937 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.501897    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:44:52.501937 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.501917    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:44:52.501937 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.501929    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bh8d5 for pod openshift-network-diagnostics/network-check-target-sm5v4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:52.502184 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.501986    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5 podName:86965e08-cf54-4abf-a78a-b377d4e2d931 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:00.501968951 +0000 UTC m=+18.294670660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bh8d5" (UniqueName: "kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5") pod "network-check-target-sm5v4" (UID: "86965e08-cf54-4abf-a78a-b377d4e2d931") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:44:52.749925 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:52.749847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:52.750112 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.749983    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:44:52.751602 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:52.751374    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:52.751602 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:52.751477    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:52.751602 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.751509    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:44:52.751602 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:52.751561    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:44:54.750165 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:54.750125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:54.750165 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:54.750149    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:54.750755 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:54.750130    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:54.750755 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:54.750246    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:44:54.750755 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:54.750322    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:44:54.750755 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:54.750420    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:44:56.753287 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:56.753210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:56.753287 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:56.753249    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:56.753765 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:56.753210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:56.753765 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:56.753334    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:44:56.753765 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:56.753437    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:44:56.753765 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:56.753519    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:44:58.752632 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:58.752599    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:44:58.753114 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:58.752704    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:44:58.753114 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:44:58.752714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:44:58.753114 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:58.752715    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:44:58.753114 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:58.752793    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:44:58.753114 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:44:58.752878    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:00.355144 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:00.355072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:00.355639 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:00.355167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:00.355639 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.355264    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:45:00.355639 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.355290    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:45:00.355639 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.355331    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret podName:ffa614d2-e094-43ee-a793-042735c1161d nodeName:}" failed. No retries permitted until 2026-09-15 12:45:16.355312441 +0000 UTC m=+34.148014163 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret") pod "global-pull-secret-syncer-9wmfs" (UID: "ffa614d2-e094-43ee-a793-042735c1161d") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:45:00.355639 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.355351    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs podName:a39d331a-f507-4475-9a80-16a20f6339b6 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:16.355341887 +0000 UTC m=+34.148043601 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs") pod "network-metrics-daemon-wdsdq" (UID: "a39d331a-f507-4475-9a80-16a20f6339b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:45:00.556376 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:00.556335    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:00.556544 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.556498    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:45:00.556544 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.556519    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:45:00.556544 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.556528    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bh8d5 for pod openshift-network-diagnostics/network-check-target-sm5v4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:45:00.556672 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.556591    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5 podName:86965e08-cf54-4abf-a78a-b377d4e2d931 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:16.55657491 +0000 UTC m=+34.349276618 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bh8d5" (UniqueName: "kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5") pod "network-check-target-sm5v4" (UID: "86965e08-cf54-4abf-a78a-b377d4e2d931") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:45:00.750049 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:00.749947    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:00.750049 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:00.749984    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:00.750049 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:00.749947    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:00.750314 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.750093    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:00.750314 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.750224    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:00.750404 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:00.750310    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:02.750818 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:02.750752    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:02.751328 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:02.750871    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:02.751328 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:02.750879    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:02.751328 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:02.750915    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:02.751328 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:02.750986    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:02.751328 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:02.751085    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:04.750239 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.749888    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:04.750796 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.749956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:04.750796 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:04.750281    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:04.750796 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.749888    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:04.750796 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:04.750352    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:04.750796 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:04.750409    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:04.849560 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.849331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" event={"ID":"40993ab6a5e5658e373011e0ad6bc90a","Type":"ContainerStarted","Data":"d218411f7fd118ed5cf3f051681b57b3457080eda6481e2d648f38b18560a9d7"}
Sep 15 12:45:04.851097 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.851067    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qk4m8" event={"ID":"34069473-d7cd-4964-b84a-ef8caa635898","Type":"ContainerStarted","Data":"a0666db6b2868e80296664653b0bbebbed3a82e514191ed6c8762729f9ae6c82"}
Sep 15 12:45:04.852696 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.852671    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lbdf9" event={"ID":"eedbe00f-27fc-4124-b091-c64793e031d3","Type":"ContainerStarted","Data":"e823bae33c68642bb32d52a5a4717bd1167724e9beaa0b648155330ecac142f1"}
Sep 15 12:45:04.854291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.854250    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nb6bf" event={"ID":"3a544b24-4e29-4395-84fe-3d2e1086f6bd","Type":"ContainerStarted","Data":"5f4f99e04044af8ffec62b62f9c4a40da67e41de7a501812a479c0d352670673"}
Sep 15 12:45:04.855795 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.855763    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vq2mc" event={"ID":"c3351e98-4182-4ebd-bccd-594fbb589096","Type":"ContainerStarted","Data":"a1da8d08f9d6b21cb4abad98868a1885e6de4016b31bb42abf3c7ae37a818867"}
Sep 15 12:45:04.859247 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.859226    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:45:04.859601 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.859581    2562 generic.go:356] "Generic (PLEG): container finished" podID="b6a65b2e-c744-46f4-b9ca-b7166c2f9107" containerID="17b1e5d86fba96ab7cbbe5067eed0cc36050503b926ba00283139cafb906d712" exitCode=1
Sep 15 12:45:04.859692 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.859642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"1e76240218bd242ea47dfd1d09391680894baa164c101872a9aca00be9d147af"}
Sep 15 12:45:04.859755 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.859707    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"0a6ede870a119c935060e3a4fbda623304f017794a4ca1910ac71be9c6e80767"}
Sep 15 12:45:04.859813 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.859756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"075336ff42b6053ea595746bc49a1e54d1f139aa41edaf84ea3b344a0bfa891d"}
Sep 15 12:45:04.859813 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.859804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"e9b84f0e260dea145e5ba096f23ec06a6c2b2dd0bcbdf6d825d3471c4d97e5c7"}
Sep 15 12:45:04.859912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.859851    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerDied","Data":"17b1e5d86fba96ab7cbbe5067eed0cc36050503b926ba00283139cafb906d712"}
Sep 15 12:45:04.859912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.859900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"a06d234db74aa891eac46aaa5ab6dd53b71270bc4618a80bdbb60c140e152621"}
Sep 15 12:45:04.861411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.861388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vb7k8" event={"ID":"1d773ad9-c704-4c02-bac5-855192af726f","Type":"ContainerStarted","Data":"9c4e1f20357264c0f8948a6ddf3a1b562088ab8e83e0b8343988ff0c924a1906"}
Sep 15 12:45:04.863230 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.863200    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb" event={"ID":"b089d305-66c1-45c1-8559-9cdc2632cc81","Type":"ContainerStarted","Data":"c55b5784f1a38eda3797e26cabcb65746542016d22dcb6a9a3a91ca653f06ce3"}
Sep 15 12:45:04.865108 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.865080    2562 generic.go:356] "Generic (PLEG): container finished" podID="2344c353-bb2a-496e-b049-844dd8edf31c" containerID="c1b7fb2fea22fa3c2e4dd2dd86b56dbb29e6602ef39dbb187818e0bb9cefd6a3" exitCode=0
Sep 15 12:45:04.865108 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.865116    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlghj" event={"ID":"2344c353-bb2a-496e-b049-844dd8edf31c","Type":"ContainerDied","Data":"c1b7fb2fea22fa3c2e4dd2dd86b56dbb29e6602ef39dbb187818e0bb9cefd6a3"}
Sep 15 12:45:04.871150 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.871097    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" podStartSLOduration=20.871081581 podStartE2EDuration="20.871081581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:44:46.839538097 +0000 UTC m=+4.632239836" watchObservedRunningTime="2026-09-15 12:45:04.871081581 +0000 UTC m=+22.663783317"
Sep 15 12:45:04.871447 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.871392    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" podStartSLOduration=20.871383122 podStartE2EDuration="20.871383122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:04.870900718 +0000 UTC m=+22.663602449" watchObservedRunningTime="2026-09-15 12:45:04.871383122 +0000 UTC m=+22.664084854"
Sep 15 12:45:04.915218 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.915182    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vq2mc" podStartSLOduration=3.186419713 podStartE2EDuration="21.915167209s" totalImagesPullingTime="18.728747496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:04.89153062 +0000 UTC m=+22.684232350" watchObservedRunningTime="2026-09-15 12:45:04.915167209 +0000 UTC m=+22.707868960"
Sep 15 12:45:04.915590 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.915568    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qk4m8" podStartSLOduration=3.160717851 podStartE2EDuration="21.915562866s" totalImagesPullingTime="18.754845015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:04.915051287 +0000 UTC m=+22.707753012" watchObservedRunningTime="2026-09-15 12:45:04.915562866 +0000 UTC m=+22.708264597"
Sep 15 12:45:04.958334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:04.958294    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vb7k8" podStartSLOduration=2.644842906 podStartE2EDuration="21.958281208s" totalImagesPullingTime="19.313438302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:04.957835863 +0000 UTC m=+22.750537593" watchObservedRunningTime="2026-09-15 12:45:04.958281208 +0000 UTC m=+22.750982940"
Sep 15 12:45:05.014242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:05.014160    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lbdf9" podStartSLOduration=7.3068077670000005 podStartE2EDuration="22.01414566s" totalImagesPullingTime="14.707337893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:05.013953772 +0000 UTC m=+22.806655503" watchObservedRunningTime="2026-09-15 12:45:05.01414566 +0000 UTC m=+22.806847390"
Sep 15 12:45:05.531253 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:05.531228    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 12:45:05.684498 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:05.684375    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T12:45:05.531250298Z","UUID":"be0cc994-d0ac-4ff6-a399-e4935455be0d","Handler":null,"Name":"","Endpoint":""}
Sep 15 12:45:05.687716 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:05.686851    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 12:45:05.687716 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:05.686902    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 12:45:05.868603 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:05.868572    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb" event={"ID":"b089d305-66c1-45c1-8559-9cdc2632cc81","Type":"ContainerStarted","Data":"cc41153653e02aa99c76a2420aa622b06fd845c64d23913135a947bc3eacdf24"}
Sep 15 12:45:05.869950 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:05.869928    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bl8tb" event={"ID":"c464e89d-901f-42bf-a59d-04f381376587","Type":"ContainerStarted","Data":"65e23406e633390210e74020368ca94d9f67c3a63f2fe0b97299bb194b70801d"}
Sep 15 12:45:05.894770 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:05.894729    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nb6bf" podStartSLOduration=4.136486587 podStartE2EDuration="22.894716636s" totalImagesPullingTime="18.758230049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:05.077792207 +0000 UTC m=+22.870493949" watchObservedRunningTime="2026-09-15 12:45:05.894716636 +0000 UTC m=+23.687418412"
Sep 15 12:45:06.750409 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:06.750179    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:06.750510 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:06.750181    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:06.750625 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:06.750529    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:06.750625 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:06.750252    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:06.750625 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:06.750580    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:06.750763 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:06.750662    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:06.875413 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:06.875381    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:45:06.875847 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:06.875751    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"5c730b9a188d9df744d535ed44c5868c553926669c08af93bd7f4d76c763f435"}
Sep 15 12:45:06.877799 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:06.877776    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb" event={"ID":"b089d305-66c1-45c1-8559-9cdc2632cc81","Type":"ContainerStarted","Data":"c5a5434ca63475bfb666543c921810e82840baf5e19ecb078ddc61d5283b2e52"}
Sep 15 12:45:06.899288 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:06.899246    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwskb" podStartSLOduration=2.621600156 podStartE2EDuration="23.899233367s" totalImagesPullingTime="21.277633211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:06.899179747 +0000 UTC m=+24.691881491" watchObservedRunningTime="2026-09-15 12:45:06.899233367 +0000 UTC m=+24.691935099"
Sep 15 12:45:06.899605 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:06.899576    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bl8tb" podStartSLOduration=5.162081647 podStartE2EDuration="23.899565291s" totalImagesPullingTime="18.737483644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:05.895228275 +0000 UTC m=+23.687930006" watchObservedRunningTime="2026-09-15 12:45:06.899565291 +0000 UTC m=+24.692267021"
Sep 15 12:45:08.755264 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.754802    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:08.755264 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:08.754926    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:08.755264 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.755172    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:08.755693 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:08.755277    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:08.755693 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.755307    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:08.755693 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:08.755401    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:08.885807 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.885744    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:45:08.886129 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.886109    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"e25585bb10849b709c992291f44f08b91215e860de8b7c1def8db26251076a4b"}
Sep 15 12:45:08.886391 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.886375    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:45:08.886469 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.886426    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:45:08.886681 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.886646    2562 scope.go:117] "RemoveContainer" containerID="17b1e5d86fba96ab7cbbe5067eed0cc36050503b926ba00283139cafb906d712"
Sep 15 12:45:08.905841 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:08.905560    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:45:09.814868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.814837    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:45:09.815526 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.815509    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:45:09.904327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.904299    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:45:09.904842 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.904797    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" event={"ID":"b6a65b2e-c744-46f4-b9ca-b7166c2f9107","Type":"ContainerStarted","Data":"bf329f121ee21cbbd98a90599d3bafbac2edbbc82ab7d41152304117c0bf85a2"}
Sep 15 12:45:09.905021 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.904982    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:45:09.905146 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.905070    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:45:09.905723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.905703    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lbdf9"
Sep 15 12:45:09.923444 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.923419    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:45:09.942769 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:09.940044    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx" podStartSLOduration=8.150867365 podStartE2EDuration="26.940009849s" totalImagesPullingTime="18.789142484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:09.939456317 +0000 UTC m=+27.732158050" watchObservedRunningTime="2026-09-15 12:45:09.940009849 +0000 UTC m=+27.732711582"
Sep 15 12:45:10.690347 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:10.690313    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sm5v4"]
Sep 15 12:45:10.690560 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:10.690485    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:10.690642 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:10.690606    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:10.693684 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:10.693628    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wdsdq"]
Sep 15 12:45:10.693805 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:10.693758    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:10.694219 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:10.694171    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:10.694372 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:10.694332    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9wmfs"]
Sep 15 12:45:10.694482 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:10.694436    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:10.694556 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:10.694532    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:11.750238 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:11.750216    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:11.750833 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:11.750324    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:11.912065 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:11.912016    2562 generic.go:356] "Generic (PLEG): container finished" podID="2344c353-bb2a-496e-b049-844dd8edf31c" containerID="371af2347012fc34797a4694251820f23e672f444708c72f99f269da97177636" exitCode=0
Sep 15 12:45:11.912239 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:11.912104    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlghj" event={"ID":"2344c353-bb2a-496e-b049-844dd8edf31c","Type":"ContainerDied","Data":"371af2347012fc34797a4694251820f23e672f444708c72f99f269da97177636"}
Sep 15 12:45:12.750961 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:12.750929    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:12.751482 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:12.751048    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:12.751482 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:12.751154    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:12.751482 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:12.751323    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:13.750115 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:13.749869    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:13.750252 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:13.750130    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:14.750014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:14.749993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:14.750421 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:14.750055    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:14.750421 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:14.750171    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:14.750421 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:14.750287    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:14.919666 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:14.919637    2562 generic.go:356] "Generic (PLEG): container finished" podID="2344c353-bb2a-496e-b049-844dd8edf31c" containerID="672c1076c35d169fc6e38395f9c4ce767cc333361db67548a36c83b0b4c1594f" exitCode=0
Sep 15 12:45:14.919802 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:14.919679    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlghj" event={"ID":"2344c353-bb2a-496e-b049-844dd8edf31c","Type":"ContainerDied","Data":"672c1076c35d169fc6e38395f9c4ce767cc333361db67548a36c83b0b4c1594f"}
Sep 15 12:45:15.750139 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:15.750107    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:15.750480 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:15.750216    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wmfs" podUID="ffa614d2-e094-43ee-a793-042735c1161d"
Sep 15 12:45:15.923820 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:15.923794    2562 generic.go:356] "Generic (PLEG): container finished" podID="2344c353-bb2a-496e-b049-844dd8edf31c" containerID="9abccd2c1c0428159e2adc0785c9f69c7ba3f3bb27cf1281f22f7e394903dc58" exitCode=0
Sep 15 12:45:15.923937 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:15.923848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlghj" event={"ID":"2344c353-bb2a-496e-b049-844dd8edf31c","Type":"ContainerDied","Data":"9abccd2c1c0428159e2adc0785c9f69c7ba3f3bb27cf1281f22f7e394903dc58"}
Sep 15 12:45:16.365770 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.365740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:16.365937 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.365787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:16.365937 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.365890    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 12:45:16.366012 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.365945    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret podName:ffa614d2-e094-43ee-a793-042735c1161d nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.36593135 +0000 UTC m=+66.158633059 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret") pod "global-pull-secret-syncer-9wmfs" (UID: "ffa614d2-e094-43ee-a793-042735c1161d") : object "kube-system"/"original-pull-secret" not registered
Sep 15 12:45:16.366012 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.365890    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:45:16.366012 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.365982    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs podName:a39d331a-f507-4475-9a80-16a20f6339b6 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.365971933 +0000 UTC m=+66.158673645 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs") pod "network-metrics-daemon-wdsdq" (UID: "a39d331a-f507-4475-9a80-16a20f6339b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 12:45:16.566649 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.566612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:16.566796 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.566772    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 12:45:16.566796 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.566790    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 12:45:16.566901 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.566803    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bh8d5 for pod openshift-network-diagnostics/network-check-target-sm5v4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:45:16.566901 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.566867    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5 podName:86965e08-cf54-4abf-a78a-b377d4e2d931 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.56684914 +0000 UTC m=+66.359550861 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bh8d5" (UniqueName: "kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5") pod "network-check-target-sm5v4" (UID: "86965e08-cf54-4abf-a78a-b377d4e2d931") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 12:45:16.750002 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.749972    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:16.750139 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.749980    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:16.750139 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.750113    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sm5v4" podUID="86965e08-cf54-4abf-a78a-b377d4e2d931"
Sep 15 12:45:16.750508 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:16.750206    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdsdq" podUID="a39d331a-f507-4475-9a80-16a20f6339b6"
Sep 15 12:45:16.912083 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.912049    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeReady"
Sep 15 12:45:16.912311 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.912287    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 12:45:16.962798 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.962405    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-qh8lp"]
Sep 15 12:45:16.964653 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.964575    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-788ffb67c9-gn56b"]
Sep 15 12:45:16.965768 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.965002    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:16.966479 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.966460    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-89fbc77c4-kzvnf"]
Sep 15 12:45:16.966653 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.966571    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:16.968074 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.968055    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"]
Sep 15 12:45:16.968171 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.968115    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:16.969284 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.969260    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 15 12:45:16.969396 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.969267    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 15 12:45:16.971109 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.971086    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-frn5n"]
Sep 15 12:45:16.971321 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.971291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:16.974826 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.974802    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-b42hd\""
Sep 15 12:45:16.974924 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.974862    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 12:45:16.975313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.975283    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 15 12:45:16.975313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.975295    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 15 12:45:16.975462 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.975349    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 15 12:45:16.975462 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.975408    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 15 12:45:16.975621 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.975293    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 15 12:45:16.975698 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.975622    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 15 12:45:16.976440 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.976423    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"]
Sep 15 12:45:16.976976 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.976955    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:16.977092 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.977074    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ht9tr\""
Sep 15 12:45:16.977264 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.977242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w89vc\""
Sep 15 12:45:16.977396 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.977379    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 15 12:45:16.977465 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.977400    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 12:45:16.977465 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.977419    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 12:45:16.977573 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.977381    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 12:45:16.979545 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.979520    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 12:45:16.979742 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.979722    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm"]
Sep 15 12:45:16.980014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.979995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:16.981059 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.981014    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 15 12:45:16.981158 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.981106    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 12:45:16.981317 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.981301    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 15 12:45:16.981745 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.981722    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:45:16.981935 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.981909    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 15 12:45:16.982018 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.981977    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fmfjc\""
Sep 15 12:45:16.982018 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.981978    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 15 12:45:16.982334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.982312    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 15 12:45:16.982419 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.982340    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm"
Sep 15 12:45:16.982745 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.982724    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-88nkn\""
Sep 15 12:45:16.984588 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.984571    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 15 12:45:16.985487 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.985465    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 15 12:45:16.987857 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.987842    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 15 12:45:16.988710 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.988323    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 15 12:45:16.988710 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.988351    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:45:16.989808 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.989792    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 15 12:45:16.991144 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.990849    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 15 12:45:16.991144 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.990373    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-c2vzm\""
Sep 15 12:45:16.991144 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.991011    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-w28t2\""
Sep 15 12:45:16.991144 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.990560    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:45:16.991980 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.991956    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"]
Sep 15 12:45:16.994449 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.993715    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"]
Sep 15 12:45:16.995535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.995512    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bhwk9"]
Sep 15 12:45:16.996161 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.996146    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:16.997808 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.997790    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"]
Sep 15 12:45:16.998674 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:16.998651    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bhwk9"
Sep 15 12:45:17.003125 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.003097    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-89fbc77c4-kzvnf"]
Sep 15 12:45:17.003216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.003140    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-frn5n"]
Sep 15 12:45:17.003216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.003162    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-788ffb67c9-gn56b"]
Sep 15 12:45:17.003216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.003188    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd"]
Sep 15 12:45:17.003398 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.003372    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.005831 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.005812    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-qh8lp"]
Sep 15 12:45:17.005933 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.005895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd"
Sep 15 12:45:17.016898 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.016866    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 15 12:45:17.017073 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017052    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 15 12:45:17.017073 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017068    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 15 12:45:17.017205 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017113    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 15 12:45:17.017261 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017205    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 15 12:45:17.017261 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017069    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-q9cxs\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017312    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017342    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017364    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017452    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017522    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7q4jr\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017611    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h4s4k\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017764    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wnhvg"]
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017837    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017909    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nrrcj\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017945    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.017985    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 12:45:17.020295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.018096    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 12:45:17.023451 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.023427    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm"]
Sep 15 12:45:17.023545 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.023460    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"]
Sep 15 12:45:17.023545 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.023482    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd"]
Sep 15 12:45:17.023545 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.023500    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bhwk9"]
Sep 15 12:45:17.023545 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.023518    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"]
Sep 15 12:45:17.023753 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.023612    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.026673 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.026510    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rtf69\""
Sep 15 12:45:17.026673 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.026576    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 12:45:17.026846 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.026832    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 12:45:17.033899 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.033877    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"]
Sep 15 12:45:17.038433 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.037998    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wnhvg"]
Sep 15 12:45:17.071449 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071427    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/518a335e-be83-4292-82e7-fd571fc39ad1-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.071554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/518a335e-be83-4292-82e7-fd571fc39ad1-tmp\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.071554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071479    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:17.071554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-trusted-ca\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.071554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95150dd3-0bc2-48d5-b06f-d38ca3f9705e-cert\") pod \"ingress-canary-bhwk9\" (UID: \"95150dd3-0bc2-48d5-b06f-d38ca3f9705e\") " pod="openshift-ingress-canary/ingress-canary-bhwk9"
Sep 15 12:45:17.071723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071553    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntrmc\" (UniqueName: \"kubernetes.io/projected/d8d2cf54-fdd9-4336-9988-12725adb2d5a-kube-api-access-ntrmc\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.071723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071602    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/518a335e-be83-4292-82e7-fd571fc39ad1-serving-cert\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.071723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071651    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-bound-sa-token\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.071723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.071723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bpwx\" (UniqueName: \"kubernetes.io/projected/255a09b1-7274-435a-8ad0-59ba327862a9-kube-api-access-6bpwx\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.071723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071717    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktvls\" (UniqueName: \"kubernetes.io/projected/7ee18f00-83f0-4177-906f-f66a46579c82-kube-api-access-ktvls\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/518a335e-be83-4292-82e7-fd571fc39ad1-snapshots\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc7dp\" (UniqueName: \"kubernetes.io/projected/861581d3-d35a-46c8-85b3-72da2998195b-kube-api-access-pc7dp\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071796    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1542bbc-a7c6-4873-8672-6607a20f7632-ca-trust-extracted\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d0130b6-3e38-41e4-a4dd-00d170040652-serving-cert\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8pmh\" (UniqueName: \"kubernetes.io/projected/5558222d-f542-4972-b7a1-a6b3425571cd-kube-api-access-d8pmh\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/255a09b1-7274-435a-8ad0-59ba327862a9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-default-certificate\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8d2cf54-fdd9-4336-9988-12725adb2d5a-serving-cert\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071953    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-stats-auth\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071968    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d0130b6-3e38-41e4-a4dd-00d170040652-trusted-ca\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.072014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.071984    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5558222d-f542-4972-b7a1-a6b3425571cd-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072080    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dscr8\" (UniqueName: \"kubernetes.io/projected/3734354c-d6c4-4df6-acd5-5b5dc7a02951-kube-api-access-dscr8\") pod \"network-check-source-6f8b768455-t6nwd\" (UID: \"3734354c-d6c4-4df6-acd5-5b5dc7a02951\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072143    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stwdd\" (UniqueName: \"kubernetes.io/projected/518a335e-be83-4292-82e7-fd571fc39ad1-kube-api-access-stwdd\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-certificates\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8d2cf54-fdd9-4336-9988-12725adb2d5a-config\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072329    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frfpq\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-kube-api-access-frfpq\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5xls\" (UniqueName: \"kubernetes.io/projected/95150dd3-0bc2-48d5-b06f-d38ca3f9705e-kube-api-access-s5xls\") pod \"ingress-canary-bhwk9\" (UID: \"95150dd3-0bc2-48d5-b06f-d38ca3f9705e\") " pod="openshift-ingress-canary/ingress-canary-bhwk9"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072483    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-image-registry-private-configuration\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/255a09b1-7274-435a-8ad0-59ba327862a9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-installation-pull-secrets\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.072558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/518a335e-be83-4292-82e7-fd571fc39ad1-service-ca-bundle\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.073092 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d0130b6-3e38-41e4-a4dd-00d170040652-config\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.073092 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqlnz\" (UniqueName: \"kubernetes.io/projected/8d0130b6-3e38-41e4-a4dd-00d170040652-kube-api-access-fqlnz\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.073092 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.072607    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn6gc\" (UniqueName: \"kubernetes.io/projected/4656aefa-4af5-49e1-bd3c-20594a1b1a88-kube-api-access-wn6gc\") pod \"volume-data-source-validator-86695b7cc7-cfdkm\" (UID: \"4656aefa-4af5-49e1-bd3c-20594a1b1a88\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm"
Sep 15 12:45:17.173910 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.173512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-bound-sa-token\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.173910 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.173748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/436023f5-a121-4823-bf16-203409532181-config-volume\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.173910 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.173781    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.173910 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.173812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bpwx\" (UniqueName: \"kubernetes.io/projected/255a09b1-7274-435a-8ad0-59ba327862a9-kube-api-access-6bpwx\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.173910 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.173838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.173910 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.173862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktvls\" (UniqueName: \"kubernetes.io/projected/7ee18f00-83f0-4177-906f-f66a46579c82-kube-api-access-ktvls\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.173910 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.173889    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.173934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/518a335e-be83-4292-82e7-fd571fc39ad1-snapshots\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.173980    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls podName:5558222d-f542-4972-b7a1-a6b3425571cd nodeName:}" failed. No retries permitted until 2026-09-15 12:45:17.673962777 +0000 UTC m=+35.466664500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b7j9r" (UID: "5558222d-f542-4972-b7a1-a6b3425571cd") : secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc7dp\" (UniqueName: \"kubernetes.io/projected/861581d3-d35a-46c8-85b3-72da2998195b-kube-api-access-pc7dp\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1542bbc-a7c6-4873-8672-6607a20f7632-ca-trust-extracted\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.174112    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.174129    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788ffb67c9-gn56b: secret "image-registry-tls" not found
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.174186    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls podName:c1542bbc-a7c6-4873-8672-6607a20f7632 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:17.674169335 +0000 UTC m=+35.466871075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls") pod "image-registry-788ffb67c9-gn56b" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632") : secret "image-registry-tls" not found
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d0130b6-3e38-41e4-a4dd-00d170040652-serving-cert\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8pmh\" (UniqueName: \"kubernetes.io/projected/5558222d-f542-4972-b7a1-a6b3425571cd-kube-api-access-d8pmh\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174311    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/255a09b1-7274-435a-8ad0-59ba327862a9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-default-certificate\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8d2cf54-fdd9-4336-9988-12725adb2d5a-serving-cert\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.174407 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-stats-auth\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d0130b6-3e38-41e4-a4dd-00d170040652-trusted-ca\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5558222d-f542-4972-b7a1-a6b3425571cd-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174516    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1542bbc-a7c6-4873-8672-6607a20f7632-ca-trust-extracted\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174528    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dscr8\" (UniqueName: \"kubernetes.io/projected/3734354c-d6c4-4df6-acd5-5b5dc7a02951-kube-api-access-dscr8\") pod \"network-check-source-6f8b768455-t6nwd\" (UID: \"3734354c-d6c4-4df6-acd5-5b5dc7a02951\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174561    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stwdd\" (UniqueName: \"kubernetes.io/projected/518a335e-be83-4292-82e7-fd571fc39ad1-kube-api-access-stwdd\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbghj\" (UniqueName: \"kubernetes.io/projected/436023f5-a121-4823-bf16-203409532181-kube-api-access-lbghj\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/518a335e-be83-4292-82e7-fd571fc39ad1-snapshots\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-certificates\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.175135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.174824    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.175562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/255a09b1-7274-435a-8ad0-59ba327862a9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.175562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8d2cf54-fdd9-4336-9988-12725adb2d5a-config\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.175562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/436023f5-a121-4823-bf16-203409532181-tmp-dir\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.175562 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.175458    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 12:45:17.175562 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.175501    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:17.675487993 +0000 UTC m=+35.468189705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : secret "router-metrics-certs-default" not found
Sep 15 12:45:17.175562 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.175531    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:17.675518015 +0000 UTC m=+35.468219742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : configmap references non-existent config key: service-ca.crt
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175634    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfpq\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-kube-api-access-frfpq\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5xls\" (UniqueName: \"kubernetes.io/projected/95150dd3-0bc2-48d5-b06f-d38ca3f9705e-kube-api-access-s5xls\") pod \"ingress-canary-bhwk9\" (UID: \"95150dd3-0bc2-48d5-b06f-d38ca3f9705e\") " pod="openshift-ingress-canary/ingress-canary-bhwk9"
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-image-registry-private-configuration\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/255a09b1-7274-435a-8ad0-59ba327862a9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-installation-pull-secrets\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175783    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/518a335e-be83-4292-82e7-fd571fc39ad1-service-ca-bundle\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d0130b6-3e38-41e4-a4dd-00d170040652-config\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqlnz\" (UniqueName: \"kubernetes.io/projected/8d0130b6-3e38-41e4-a4dd-00d170040652-kube-api-access-fqlnz\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.175868 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175844    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn6gc\" (UniqueName: \"kubernetes.io/projected/4656aefa-4af5-49e1-bd3c-20594a1b1a88-kube-api-access-wn6gc\") pod \"volume-data-source-validator-86695b7cc7-cfdkm\" (UID: \"4656aefa-4af5-49e1-bd3c-20594a1b1a88\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm"
Sep 15 12:45:17.176324 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/518a335e-be83-4292-82e7-fd571fc39ad1-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.176324 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/518a335e-be83-4292-82e7-fd571fc39ad1-tmp\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.176324 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d0130b6-3e38-41e4-a4dd-00d170040652-trusted-ca\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.176324 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175926    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:17.176324 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.175947    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-trusted-ca\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.176324 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.176115    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 12:45:17.176324 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.176183    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-certificates\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.176646 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.176397    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls podName:861581d3-d35a-46c8-85b3-72da2998195b nodeName:}" failed. No retries permitted until 2026-09-15 12:45:17.676381664 +0000 UTC m=+35.469083374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-52749" (UID: "861581d3-d35a-46c8-85b3-72da2998195b") : secret "samples-operator-tls" not found
Sep 15 12:45:17.176646 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.176430    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5558222d-f542-4972-b7a1-a6b3425571cd-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.176646 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.176485    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95150dd3-0bc2-48d5-b06f-d38ca3f9705e-cert\") pod \"ingress-canary-bhwk9\" (UID: \"95150dd3-0bc2-48d5-b06f-d38ca3f9705e\") " pod="openshift-ingress-canary/ingress-canary-bhwk9"
Sep 15 12:45:17.176646 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.176513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntrmc\" (UniqueName: \"kubernetes.io/projected/d8d2cf54-fdd9-4336-9988-12725adb2d5a-kube-api-access-ntrmc\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.176646 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.176537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/518a335e-be83-4292-82e7-fd571fc39ad1-serving-cert\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.177346 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.177268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8d2cf54-fdd9-4336-9988-12725adb2d5a-config\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.177653 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.177590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/518a335e-be83-4292-82e7-fd571fc39ad1-service-ca-bundle\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.178117 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.178093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-trusted-ca\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.178117 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.178099    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/518a335e-be83-4292-82e7-fd571fc39ad1-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.178546 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.178522    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/518a335e-be83-4292-82e7-fd571fc39ad1-tmp\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.179200 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.179162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d0130b6-3e38-41e4-a4dd-00d170040652-config\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.179670 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.179646    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/255a09b1-7274-435a-8ad0-59ba327862a9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.180927 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.180445    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8d2cf54-fdd9-4336-9988-12725adb2d5a-serving-cert\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.180927 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.180803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d0130b6-3e38-41e4-a4dd-00d170040652-serving-cert\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.180927 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.180897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-stats-auth\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.181141 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.180973    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/518a335e-be83-4292-82e7-fd571fc39ad1-serving-cert\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.181141 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.180991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-default-certificate\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.181531 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.181472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-image-registry-private-configuration\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.181747 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.181724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-installation-pull-secrets\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.182768 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.182745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95150dd3-0bc2-48d5-b06f-d38ca3f9705e-cert\") pod \"ingress-canary-bhwk9\" (UID: \"95150dd3-0bc2-48d5-b06f-d38ca3f9705e\") " pod="openshift-ingress-canary/ingress-canary-bhwk9"
Sep 15 12:45:17.185747 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.185634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stwdd\" (UniqueName: \"kubernetes.io/projected/518a335e-be83-4292-82e7-fd571fc39ad1-kube-api-access-stwdd\") pod \"insights-operator-5844b856db-qh8lp\" (UID: \"518a335e-be83-4292-82e7-fd571fc39ad1\") " pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.190061 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.190021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8pmh\" (UniqueName: \"kubernetes.io/projected/5558222d-f542-4972-b7a1-a6b3425571cd-kube-api-access-d8pmh\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.190640 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.190616    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc7dp\" (UniqueName: \"kubernetes.io/projected/861581d3-d35a-46c8-85b3-72da2998195b-kube-api-access-pc7dp\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:17.191955 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.191306    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bpwx\" (UniqueName: \"kubernetes.io/projected/255a09b1-7274-435a-8ad0-59ba327862a9-kube-api-access-6bpwx\") pod \"kube-storage-version-migrator-operator-6c8f84f897-bz6pm\" (UID: \"255a09b1-7274-435a-8ad0-59ba327862a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.191955 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.191553    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntrmc\" (UniqueName: \"kubernetes.io/projected/d8d2cf54-fdd9-4336-9988-12725adb2d5a-kube-api-access-ntrmc\") pod \"service-ca-operator-cf7c776b7-kdnnq\" (UID: \"d8d2cf54-fdd9-4336-9988-12725adb2d5a\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.191955 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.191903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frfpq\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-kube-api-access-frfpq\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.192172 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.191968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktvls\" (UniqueName: \"kubernetes.io/projected/7ee18f00-83f0-4177-906f-f66a46579c82-kube-api-access-ktvls\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.192941 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.192860    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn6gc\" (UniqueName: \"kubernetes.io/projected/4656aefa-4af5-49e1-bd3c-20594a1b1a88-kube-api-access-wn6gc\") pod \"volume-data-source-validator-86695b7cc7-cfdkm\" (UID: \"4656aefa-4af5-49e1-bd3c-20594a1b1a88\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm"
Sep 15 12:45:17.193878 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.193832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-bound-sa-token\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.194045 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.194006    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dscr8\" (UniqueName: \"kubernetes.io/projected/3734354c-d6c4-4df6-acd5-5b5dc7a02951-kube-api-access-dscr8\") pod \"network-check-source-6f8b768455-t6nwd\" (UID: \"3734354c-d6c4-4df6-acd5-5b5dc7a02951\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd"
Sep 15 12:45:17.194453 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.194420    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqlnz\" (UniqueName: \"kubernetes.io/projected/8d0130b6-3e38-41e4-a4dd-00d170040652-kube-api-access-fqlnz\") pod \"console-operator-696884fb9c-frn5n\" (UID: \"8d0130b6-3e38-41e4-a4dd-00d170040652\") " pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.195083 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.195061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5xls\" (UniqueName: \"kubernetes.io/projected/95150dd3-0bc2-48d5-b06f-d38ca3f9705e-kube-api-access-s5xls\") pod \"ingress-canary-bhwk9\" (UID: \"95150dd3-0bc2-48d5-b06f-d38ca3f9705e\") " pod="openshift-ingress-canary/ingress-canary-bhwk9"
Sep 15 12:45:17.277997 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.277920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/436023f5-a121-4823-bf16-203409532181-config-volume\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.278245 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.278008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.278245 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.278093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbghj\" (UniqueName: \"kubernetes.io/projected/436023f5-a121-4823-bf16-203409532181-kube-api-access-lbghj\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.278245 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.278133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/436023f5-a121-4823-bf16-203409532181-tmp-dir\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.278245 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.278140    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:45:17.278245 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.278205    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls podName:436023f5-a121-4823-bf16-203409532181 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:17.778185812 +0000 UTC m=+35.570887523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls") pod "dns-default-wnhvg" (UID: "436023f5-a121-4823-bf16-203409532181") : secret "dns-default-metrics-tls" not found
Sep 15 12:45:17.278584 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.278417    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/436023f5-a121-4823-bf16-203409532181-tmp-dir\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.278584 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.278539    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/436023f5-a121-4823-bf16-203409532181-config-volume\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.282867 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.282833    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-qh8lp"
Sep 15 12:45:17.286722 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.286700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbghj\" (UniqueName: \"kubernetes.io/projected/436023f5-a121-4823-bf16-203409532181-kube-api-access-lbghj\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.316547 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.316525    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:17.333495 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.333401    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm"
Sep 15 12:45:17.343352 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.343095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"
Sep 15 12:45:17.350244 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.349973    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bhwk9"
Sep 15 12:45:17.356645 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.356508    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"
Sep 15 12:45:17.366308 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.366280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd"
Sep 15 12:45:17.576902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.576727    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-qh8lp"]
Sep 15 12:45:17.579836 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:17.579806    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod518a335e_be83_4292_82e7_fd571fc39ad1.slice/crio-5276964e4cc9ceaca648665e472e3acf8a41d092002934cb7f6eaa160c4ce11f WatchSource:0}: Error finding container 5276964e4cc9ceaca648665e472e3acf8a41d092002934cb7f6eaa160c4ce11f: Status 404 returned error can't find the container with id 5276964e4cc9ceaca648665e472e3acf8a41d092002934cb7f6eaa160c4ce11f
Sep 15 12:45:17.606602 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.606577    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-frn5n"]
Sep 15 12:45:17.608831 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:17.608807    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0130b6_3e38_41e4_a4dd_00d170040652.slice/crio-1c1f4d0aa7c548d325a92ddef8a7da40489e3d3fa1d92ff2e4a4ccb925b2be33 WatchSource:0}: Error finding container 1c1f4d0aa7c548d325a92ddef8a7da40489e3d3fa1d92ff2e4a4ccb925b2be33: Status 404 returned error can't find the container with id 1c1f4d0aa7c548d325a92ddef8a7da40489e3d3fa1d92ff2e4a4ccb925b2be33
Sep 15 12:45:17.646456 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.646350    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm"]
Sep 15 12:45:17.649075 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:17.649043    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4656aefa_4af5_49e1_bd3c_20594a1b1a88.slice/crio-392f8fc1fab74a94abd82176063c286803034ae5214e32746a103376f3ea2af4 WatchSource:0}: Error finding container 392f8fc1fab74a94abd82176063c286803034ae5214e32746a103376f3ea2af4: Status 404 returned error can't find the container with id 392f8fc1fab74a94abd82176063c286803034ae5214e32746a103376f3ea2af4
Sep 15 12:45:17.652207 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.651917    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm"]
Sep 15 12:45:17.654354 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:17.654330    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod255a09b1_7274_435a_8ad0_59ba327862a9.slice/crio-757546b8801e6f4d0527af99061fdf487012a503321d12ece6cfd3f971741b85 WatchSource:0}: Error finding container 757546b8801e6f4d0527af99061fdf487012a503321d12ece6cfd3f971741b85: Status 404 returned error can't find the container with id 757546b8801e6f4d0527af99061fdf487012a503321d12ece6cfd3f971741b85
Sep 15 12:45:17.663381 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.663357    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bhwk9"]
Sep 15 12:45:17.666169 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:17.666144    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95150dd3_0bc2_48d5_b06f_d38ca3f9705e.slice/crio-472b6721d7285de8433136020978b085f59b9f3035de5eb8735b8709a28f9abd WatchSource:0}: Error finding container 472b6721d7285de8433136020978b085f59b9f3035de5eb8735b8709a28f9abd: Status 404 returned error can't find the container with id 472b6721d7285de8433136020978b085f59b9f3035de5eb8735b8709a28f9abd
Sep 15 12:45:17.682354 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.682334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.682453 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.682410    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:17.682510 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682460    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:18.682439018 +0000 UTC m=+36.475140747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : configmap references non-existent config key: service-ca.crt
Sep 15 12:45:17.682510 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.682493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:17.682620 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682528    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 12:45:17.682620 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682576    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls podName:861581d3-d35a-46c8-85b3-72da2998195b nodeName:}" failed. No retries permitted until 2026-09-15 12:45:18.68256042 +0000 UTC m=+36.475262143 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-52749" (UID: "861581d3-d35a-46c8-85b3-72da2998195b") : secret "samples-operator-tls" not found
Sep 15 12:45:17.682620 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.682528    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:17.682620 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682601    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 12:45:17.682620 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682613    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788ffb67c9-gn56b: secret "image-registry-tls" not found
Sep 15 12:45:17.682842 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.682624    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:17.682842 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682647    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls podName:c1542bbc-a7c6-4873-8672-6607a20f7632 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:18.682634669 +0000 UTC m=+36.475336379 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls") pod "image-registry-788ffb67c9-gn56b" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632") : secret "image-registry-tls" not found
Sep 15 12:45:17.682842 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682692    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 12:45:17.682842 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682712    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:17.682842 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682726    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:18.682714787 +0000 UTC m=+36.475416510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : secret "router-metrics-certs-default" not found
Sep 15 12:45:17.682842 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.682809    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls podName:5558222d-f542-4972-b7a1-a6b3425571cd nodeName:}" failed. No retries permitted until 2026-09-15 12:45:18.682796227 +0000 UTC m=+36.475497938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b7j9r" (UID: "5558222d-f542-4972-b7a1-a6b3425571cd") : secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:17.687536 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.687516    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd"]
Sep 15 12:45:17.690500 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.690481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq"]
Sep 15 12:45:17.690687 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:17.690665    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3734354c_d6c4_4df6_acd5_5b5dc7a02951.slice/crio-7cef4e5b593c09ff2aeb3116e69e973667e00be06ae609276a33b4c7f9b265a4 WatchSource:0}: Error finding container 7cef4e5b593c09ff2aeb3116e69e973667e00be06ae609276a33b4c7f9b265a4: Status 404 returned error can't find the container with id 7cef4e5b593c09ff2aeb3116e69e973667e00be06ae609276a33b4c7f9b265a4
Sep 15 12:45:17.694423 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:17.694398    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d2cf54_fdd9_4336_9988_12725adb2d5a.slice/crio-d4d248309890f0ac0dc04f7c201c2379f9c60aa3addc8da41fdcfb9ae4012efb WatchSource:0}: Error finding container d4d248309890f0ac0dc04f7c201c2379f9c60aa3addc8da41fdcfb9ae4012efb: Status 404 returned error can't find the container with id d4d248309890f0ac0dc04f7c201c2379f9c60aa3addc8da41fdcfb9ae4012efb
Sep 15 12:45:17.750089 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.750067    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:17.753085 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.753068    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 12:45:17.784024 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.783997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:17.784163 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.784133    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:45:17.784262 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:17.784204    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls podName:436023f5-a121-4823-bf16-203409532181 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:18.784185886 +0000 UTC m=+36.576887613 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls") pod "dns-default-wnhvg" (UID: "436023f5-a121-4823-bf16-203409532181") : secret "dns-default-metrics-tls" not found
Sep 15 12:45:17.930353 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.930320    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd" event={"ID":"3734354c-d6c4-4df6-acd5-5b5dc7a02951","Type":"ContainerStarted","Data":"7cef4e5b593c09ff2aeb3116e69e973667e00be06ae609276a33b4c7f9b265a4"}
Sep 15 12:45:17.931663 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.931636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm" event={"ID":"4656aefa-4af5-49e1-bd3c-20594a1b1a88","Type":"ContainerStarted","Data":"392f8fc1fab74a94abd82176063c286803034ae5214e32746a103376f3ea2af4"}
Sep 15 12:45:17.932760 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.932738    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq" event={"ID":"d8d2cf54-fdd9-4336-9988-12725adb2d5a","Type":"ContainerStarted","Data":"d4d248309890f0ac0dc04f7c201c2379f9c60aa3addc8da41fdcfb9ae4012efb"}
Sep 15 12:45:17.933948 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.933920    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bhwk9" event={"ID":"95150dd3-0bc2-48d5-b06f-d38ca3f9705e","Type":"ContainerStarted","Data":"472b6721d7285de8433136020978b085f59b9f3035de5eb8735b8709a28f9abd"}
Sep 15 12:45:17.935156 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.935134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm" event={"ID":"255a09b1-7274-435a-8ad0-59ba327862a9","Type":"ContainerStarted","Data":"757546b8801e6f4d0527af99061fdf487012a503321d12ece6cfd3f971741b85"}
Sep 15 12:45:17.936461 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.936425    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qh8lp" event={"ID":"518a335e-be83-4292-82e7-fd571fc39ad1","Type":"ContainerStarted","Data":"5276964e4cc9ceaca648665e472e3acf8a41d092002934cb7f6eaa160c4ce11f"}
Sep 15 12:45:17.937617 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:17.937587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-frn5n" event={"ID":"8d0130b6-3e38-41e4-a4dd-00d170040652","Type":"ContainerStarted","Data":"1c1f4d0aa7c548d325a92ddef8a7da40489e3d3fa1d92ff2e4a4ccb925b2be33"}
Sep 15 12:45:18.693721 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.693683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:18.693898 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.693738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:18.693898 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.693786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:18.693898 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.693886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:18.694092 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.693945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:18.694156 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694110    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 12:45:18.694209 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694185    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls podName:861581d3-d35a-46c8-85b3-72da2998195b nodeName:}" failed. No retries permitted until 2026-09-15 12:45:20.694165353 +0000 UTC m=+38.486867075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-52749" (UID: "861581d3-d35a-46c8-85b3-72da2998195b") : secret "samples-operator-tls" not found
Sep 15 12:45:18.694599 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694581    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:18.694671 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694638    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls podName:5558222d-f542-4972-b7a1-a6b3425571cd nodeName:}" failed. No retries permitted until 2026-09-15 12:45:20.694622027 +0000 UTC m=+38.487323750 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b7j9r" (UID: "5558222d-f542-4972-b7a1-a6b3425571cd") : secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:18.694728 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694706    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 12:45:18.694728 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694717    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788ffb67c9-gn56b: secret "image-registry-tls" not found
Sep 15 12:45:18.694830 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694750    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls podName:c1542bbc-a7c6-4873-8672-6607a20f7632 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:20.694739036 +0000 UTC m=+38.487440749 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls") pod "image-registry-788ffb67c9-gn56b" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632") : secret "image-registry-tls" not found
Sep 15 12:45:18.694830 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694804    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 12:45:18.694936 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694832    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:20.694822979 +0000 UTC m=+38.487524691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : secret "router-metrics-certs-default" not found
Sep 15 12:45:18.694936 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.694894    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:20.694884642 +0000 UTC m=+38.487586366 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : configmap references non-existent config key: service-ca.crt
Sep 15 12:45:18.756987 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.756122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:18.756987 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.756533    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:18.765812 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.761219    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 12:45:18.765812 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.761347    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w5cgx\""
Sep 15 12:45:18.765812 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.761445    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z9dbl\""
Sep 15 12:45:18.795072 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:18.795047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:18.795257 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.795240    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:45:18.795328 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:18.795301    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls podName:436023f5-a121-4823-bf16-203409532181 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:20.795283315 +0000 UTC m=+38.587985038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls") pod "dns-default-wnhvg" (UID: "436023f5-a121-4823-bf16-203409532181") : secret "dns-default-metrics-tls" not found
Sep 15 12:45:20.715657 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:20.715616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:20.715670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:20.715709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.715795    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.715816    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788ffb67c9-gn56b: secret "image-registry-tls" not found
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:20.715800    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.715792    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.715882    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls podName:c1542bbc-a7c6-4873-8672-6607a20f7632 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:24.71586106 +0000 UTC m=+42.508562786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls") pod "image-registry-788ffb67c9-gn56b" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632") : secret "image-registry-tls" not found
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.715902    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.715936    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:24.715923849 +0000 UTC m=+42.508625564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : configmap references non-existent config key: service-ca.crt
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.715954    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls podName:5558222d-f542-4972-b7a1-a6b3425571cd nodeName:}" failed. No retries permitted until 2026-09-15 12:45:24.715946422 +0000 UTC m=+42.508648132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b7j9r" (UID: "5558222d-f542-4972-b7a1-a6b3425571cd") : secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.715995    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:24.715976228 +0000 UTC m=+42.508677954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : secret "router-metrics-certs-default" not found
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:20.716095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.716212    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 12:45:20.716261 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.716256    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls podName:861581d3-d35a-46c8-85b3-72da2998195b nodeName:}" failed. No retries permitted until 2026-09-15 12:45:24.716243751 +0000 UTC m=+42.508945479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-52749" (UID: "861581d3-d35a-46c8-85b3-72da2998195b") : secret "samples-operator-tls" not found
Sep 15 12:45:20.817517 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:20.817473    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:20.817701 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.817668    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:45:20.817783 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:20.817746    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls podName:436023f5-a121-4823-bf16-203409532181 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:24.817725879 +0000 UTC m=+42.610427611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls") pod "dns-default-wnhvg" (UID: "436023f5-a121-4823-bf16-203409532181") : secret "dns-default-metrics-tls" not found
Sep 15 12:45:24.756542 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:24.756511    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:24.756563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:24.756602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:24.756630    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:24.756662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756682    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:32.756662835 +0000 UTC m=+50.549364564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : configmap references non-existent config key: service-ca.crt
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756755    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756769    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788ffb67c9-gn56b: secret "image-registry-tls" not found
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756769    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756805    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls podName:c1542bbc-a7c6-4873-8672-6607a20f7632 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:32.756793213 +0000 UTC m=+50.549494936 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls") pod "image-registry-788ffb67c9-gn56b" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632") : secret "image-registry-tls" not found
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756835    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls podName:5558222d-f542-4972-b7a1-a6b3425571cd nodeName:}" failed. No retries permitted until 2026-09-15 12:45:32.756815015 +0000 UTC m=+50.549516738 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b7j9r" (UID: "5558222d-f542-4972-b7a1-a6b3425571cd") : secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756768    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756871    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:32.756860477 +0000 UTC m=+50.549562203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : secret "router-metrics-certs-default" not found
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756877    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 12:45:24.756999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.756949    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls podName:861581d3-d35a-46c8-85b3-72da2998195b nodeName:}" failed. No retries permitted until 2026-09-15 12:45:32.756918931 +0000 UTC m=+50.549620644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-52749" (UID: "861581d3-d35a-46c8-85b3-72da2998195b") : secret "samples-operator-tls" not found
Sep 15 12:45:24.858153 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:24.858125    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:24.858306 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.858253    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:45:24.858306 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:24.858303    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls podName:436023f5-a121-4823-bf16-203409532181 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:32.858290047 +0000 UTC m=+50.650991769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls") pod "dns-default-wnhvg" (UID: "436023f5-a121-4823-bf16-203409532181") : secret "dns-default-metrics-tls" not found
Sep 15 12:45:26.964713 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:26.964684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm" event={"ID":"4656aefa-4af5-49e1-bd3c-20594a1b1a88","Type":"ContainerStarted","Data":"b7ed3da3ca78355736461fc85d4776bc243c6e428a07d500e6a096b4031622fb"}
Sep 15 12:45:27.975578 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.975471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bhwk9" event={"ID":"95150dd3-0bc2-48d5-b06f-d38ca3f9705e","Type":"ContainerStarted","Data":"93748d78f8b7fc9dc28f9f4b18c44993565bd3a09e4ff0a68bf3fbee720e562d"}
Sep 15 12:45:27.977471 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.977443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm" event={"ID":"255a09b1-7274-435a-8ad0-59ba327862a9","Type":"ContainerStarted","Data":"d6ba220bd494c375113b0c0784034b768a4f4e2cad5089e6906c00a5e057bb77"}
Sep 15 12:45:27.979159 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.979132    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qh8lp" event={"ID":"518a335e-be83-4292-82e7-fd571fc39ad1","Type":"ContainerStarted","Data":"24a4fd0cd91a994ca5e43f06ee06cb8d5c1f75895c4696b4e30cad2b2f2fb921"}
Sep 15 12:45:27.981269 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.981248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/0.log"
Sep 15 12:45:27.981372 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.981289    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d0130b6-3e38-41e4-a4dd-00d170040652" containerID="3517045ff2a5cf090ec23bafff42cb5ef3939ccd2984fdaf326af25de7d5852b" exitCode=255
Sep 15 12:45:27.981372 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.981362    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-frn5n" event={"ID":"8d0130b6-3e38-41e4-a4dd-00d170040652","Type":"ContainerDied","Data":"3517045ff2a5cf090ec23bafff42cb5ef3939ccd2984fdaf326af25de7d5852b"}
Sep 15 12:45:27.981846 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.981828    2562 scope.go:117] "RemoveContainer" containerID="3517045ff2a5cf090ec23bafff42cb5ef3939ccd2984fdaf326af25de7d5852b"
Sep 15 12:45:27.983291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.983270    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd" event={"ID":"3734354c-d6c4-4df6-acd5-5b5dc7a02951","Type":"ContainerStarted","Data":"7e08860d76e180909b92a4635eb796e3562ba911190cf08efc59fb1ce21dd577"}
Sep 15 12:45:27.987578 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.987554    2562 generic.go:356] "Generic (PLEG): container finished" podID="2344c353-bb2a-496e-b049-844dd8edf31c" containerID="9365d53a622caf93bdd020b01c284ae6c1059fea73649d728dec2fa768dd54c6" exitCode=0
Sep 15 12:45:27.987686 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.987627    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlghj" event={"ID":"2344c353-bb2a-496e-b049-844dd8edf31c","Type":"ContainerDied","Data":"9365d53a622caf93bdd020b01c284ae6c1059fea73649d728dec2fa768dd54c6"}
Sep 15 12:45:27.989224 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.989202    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq" event={"ID":"d8d2cf54-fdd9-4336-9988-12725adb2d5a","Type":"ContainerStarted","Data":"061b0ced445102de68932ede63c05cd1088adcb6c4a445acd67929bfdd1589db"}
Sep 15 12:45:27.997524 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:27.997484    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bhwk9" podStartSLOduration=2.847508752 podStartE2EDuration="11.997471919s" totalImagesPullingTime="9.149963167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:27.996671996 +0000 UTC m=+45.789373727" watchObservedRunningTime="2026-09-15 12:45:27.997471919 +0000 UTC m=+45.790173650"
Sep 15 12:45:28.090723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.089949    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm" podStartSLOduration=19.896932442 podStartE2EDuration="29.089928979s" totalImagesPullingTime="9.192996537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:28.06210434 +0000 UTC m=+45.854806071" watchObservedRunningTime="2026-09-15 12:45:28.089928979 +0000 UTC m=+45.882630714"
Sep 15 12:45:28.114258 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.114200    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cfdkm" podStartSLOduration=23.947790163 podStartE2EDuration="33.114179177s" totalImagesPullingTime="9.166389014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:28.111777967 +0000 UTC m=+45.904479700" watchObservedRunningTime="2026-09-15 12:45:28.114179177 +0000 UTC m=+45.906880909"
Sep 15 12:45:28.114558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.114524    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6nwd" podStartSLOduration=16.962667643 podStartE2EDuration="26.114517835s" totalImagesPullingTime="9.151850192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:28.090846105 +0000 UTC m=+45.883547838" watchObservedRunningTime="2026-09-15 12:45:28.114517835 +0000 UTC m=+45.907219562"
Sep 15 12:45:28.142551 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.142470    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-qh8lp" podStartSLOduration=23.933057288 podStartE2EDuration="33.142459234s" totalImagesPullingTime="9.209401946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:28.141479406 +0000 UTC m=+45.934181139" watchObservedRunningTime="2026-09-15 12:45:28.142459234 +0000 UTC m=+45.935160964"
Sep 15 12:45:28.206842 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.206558    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq" podStartSLOduration=17.084703001 podStartE2EDuration="26.206539322s" totalImagesPullingTime="9.121836321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:28.206228501 +0000 UTC m=+45.998930233" watchObservedRunningTime="2026-09-15 12:45:28.206539322 +0000 UTC m=+45.999241060"
Sep 15 12:45:28.358497 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.358465    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4jx7t"]
Sep 15 12:45:28.374146 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.374103    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4jx7t"]
Sep 15 12:45:28.374318 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.374241    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.377250 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.377227    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 12:45:28.377373 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.377237    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmbfk\""
Sep 15 12:45:28.377589 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.377571    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 12:45:28.392154 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.392130    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd9dw\" (UniqueName: \"kubernetes.io/projected/9efbb847-1884-4528-8390-64cb81e1f483-kube-api-access-gd9dw\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.392262 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.392244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9efbb847-1884-4528-8390-64cb81e1f483-data-volume\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.392327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.392288    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9efbb847-1884-4528-8390-64cb81e1f483-crio-socket\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.392389 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.392344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.392502 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.392457    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.493316 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.493278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd9dw\" (UniqueName: \"kubernetes.io/projected/9efbb847-1884-4528-8390-64cb81e1f483-kube-api-access-gd9dw\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.493490 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.493344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9efbb847-1884-4528-8390-64cb81e1f483-data-volume\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.493490 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.493392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9efbb847-1884-4528-8390-64cb81e1f483-crio-socket\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.493490 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.493451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.493490 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.493467    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9efbb847-1884-4528-8390-64cb81e1f483-crio-socket\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.493660 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.493530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.493660 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:28.493567    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:45:28.493660 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:28.493640    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:28.493660 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:28.493651    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:28.993633856 +0000 UTC m=+46.786335587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : configmap "kube-rbac-proxy" not found
Sep 15 12:45:28.493827 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:28.493704    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:28.993691711 +0000 UTC m=+46.786393423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:28.493827 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.493735    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9efbb847-1884-4528-8390-64cb81e1f483-data-volume\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.513078 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.513052    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd9dw\" (UniqueName: \"kubernetes.io/projected/9efbb847-1884-4528-8390-64cb81e1f483-kube-api-access-gd9dw\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.995992 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.995958    2562 generic.go:356] "Generic (PLEG): container finished" podID="2344c353-bb2a-496e-b049-844dd8edf31c" containerID="2d2b7e60243a5903f83ada4ab4d9e6e5b66361c2294b4c74de994a29b9dfb751" exitCode=0
Sep 15 12:45:28.996456 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.996056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlghj" event={"ID":"2344c353-bb2a-496e-b049-844dd8edf31c","Type":"ContainerDied","Data":"2d2b7e60243a5903f83ada4ab4d9e6e5b66361c2294b4c74de994a29b9dfb751"}
Sep 15 12:45:28.997022 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.996976    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.997110 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.997090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:28.997165 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:28.997112    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:45:28.997220 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:28.997170    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:29.997155979 +0000 UTC m=+47.789857702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : configmap "kube-rbac-proxy" not found
Sep 15 12:45:28.997220 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:28.997195    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:28.997334 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:28.997237    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:29.997223475 +0000 UTC m=+47.789925198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:28.997798 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.997781    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:45:28.998220 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.998205    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/0.log"
Sep 15 12:45:28.998291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.998237    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d0130b6-3e38-41e4-a4dd-00d170040652" containerID="c9a6a4a4d0e8579feb182b304dd4fcdb9b3050d7cfd294333e2c6fb3da146600" exitCode=255
Sep 15 12:45:29.002878 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.998605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-frn5n" event={"ID":"8d0130b6-3e38-41e4-a4dd-00d170040652","Type":"ContainerDied","Data":"c9a6a4a4d0e8579feb182b304dd4fcdb9b3050d7cfd294333e2c6fb3da146600"}
Sep 15 12:45:29.002878 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:28.998684    2562 scope.go:117] "RemoveContainer" containerID="3517045ff2a5cf090ec23bafff42cb5ef3939ccd2984fdaf326af25de7d5852b"
Sep 15 12:45:29.004143 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:29.004124    2562 scope.go:117] "RemoveContainer" containerID="c9a6a4a4d0e8579feb182b304dd4fcdb9b3050d7cfd294333e2c6fb3da146600"
Sep 15 12:45:29.004315 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:29.004293    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-frn5n_openshift-console-operator(8d0130b6-3e38-41e4-a4dd-00d170040652)\"" pod="openshift-console-operator/console-operator-696884fb9c-frn5n" podUID="8d0130b6-3e38-41e4-a4dd-00d170040652"
Sep 15 12:45:29.601992 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:29.601963    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nb6bf_3a544b24-4e29-4395-84fe-3d2e1086f6bd/dns-node-resolver/0.log"
Sep 15 12:45:30.006155 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:30.006128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:30.006619 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:30.006255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:30.006619 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:30.006284    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:30.006619 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:30.006351    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:45:30.006619 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:30.006366    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:32.006347338 +0000 UTC m=+49.799049049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:30.006619 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:30.006410    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:32.006398615 +0000 UTC m=+49.799100338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : configmap "kube-rbac-proxy" not found
Sep 15 12:45:30.008019 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:30.007993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlghj" event={"ID":"2344c353-bb2a-496e-b049-844dd8edf31c","Type":"ContainerStarted","Data":"607c1976a02c4099f546eb492e496130e1593e63edfd93f693d00e3a4f14e9bd"}
Sep 15 12:45:30.009640 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:30.009619    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:45:30.009958 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:30.009941    2562 scope.go:117] "RemoveContainer" containerID="c9a6a4a4d0e8579feb182b304dd4fcdb9b3050d7cfd294333e2c6fb3da146600"
Sep 15 12:45:30.010132 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:30.010111    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-frn5n_openshift-console-operator(8d0130b6-3e38-41e4-a4dd-00d170040652)\"" pod="openshift-console-operator/console-operator-696884fb9c-frn5n" podUID="8d0130b6-3e38-41e4-a4dd-00d170040652"
Sep 15 12:45:30.029761 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:30.029721    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vlghj" podStartSLOduration=7.310031346 podStartE2EDuration="47.029707699s" totalImagesPullingTime="39.719676353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:30.02868175 +0000 UTC m=+47.821383500" watchObservedRunningTime="2026-09-15 12:45:30.029707699 +0000 UTC m=+47.822409475"
Sep 15 12:45:30.801017 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:30.800988    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vq2mc_c3351e98-4182-4ebd-bccd-594fbb589096/node-ca/0.log"
Sep 15 12:45:31.211697 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:31.211670    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bhwk9_95150dd3-0bc2-48d5-b06f-d38ca3f9705e/serve-healthcheck-canary/0.log"
Sep 15 12:45:32.024127 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:32.024095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:32.024301 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:32.024158    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:32.024301 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.024254    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:32.024301 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.024272    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 12:45:32.024414 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.024311    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:36.024297416 +0000 UTC m=+53.816999125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:32.024414 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.024341    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:36.024327123 +0000 UTC m=+53.817028845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : configmap "kube-rbac-proxy" not found
Sep 15 12:45:32.828475 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:32.828439    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:32.828475 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:32.828478    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:32.828520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.828586    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.828640    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.828651    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.828669    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788ffb67c9-gn56b: secret "image-registry-tls" not found
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.828657    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls podName:5558222d-f542-4972-b7a1-a6b3425571cd nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.828637513 +0000 UTC m=+66.621339234 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b7j9r" (UID: "5558222d-f542-4972-b7a1-a6b3425571cd") : secret "cluster-monitoring-operator-tls" not found
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:32.828749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.828798    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.828785658 +0000 UTC m=+66.621487368 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : secret "router-metrics-certs-default" not found
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.828808    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls podName:c1542bbc-a7c6-4873-8672-6607a20f7632 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.828802716 +0000 UTC m=+66.621504425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls") pod "image-registry-788ffb67c9-gn56b" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632") : secret "image-registry-tls" not found
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.828862    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle podName:7ee18f00-83f0-4177-906f-f66a46579c82 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.828845231 +0000 UTC m=+66.621546946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle") pod "router-default-89fbc77c4-kzvnf" (UID: "7ee18f00-83f0-4177-906f-f66a46579c82") : configmap references non-existent config key: service-ca.crt
Sep 15 12:45:32.828920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:32.828894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:32.829351 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.829010    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 12:45:32.829351 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.829080    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls podName:861581d3-d35a-46c8-85b3-72da2998195b nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.829066948 +0000 UTC m=+66.621768662 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-52749" (UID: "861581d3-d35a-46c8-85b3-72da2998195b") : secret "samples-operator-tls" not found
Sep 15 12:45:32.930249 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:32.930220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:32.930385 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.930355    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 12:45:32.930423 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:32.930403    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls podName:436023f5-a121-4823-bf16-203409532181 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:48.930391669 +0000 UTC m=+66.723093377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls") pod "dns-default-wnhvg" (UID: "436023f5-a121-4823-bf16-203409532181") : secret "dns-default-metrics-tls" not found
Sep 15 12:45:36.055984 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:36.055946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:36.056487 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:36.056055    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:36.056487 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:36.056190    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:36.056487 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:36.056249    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls podName:9efbb847-1884-4528-8390-64cb81e1f483 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:44.056233778 +0000 UTC m=+61.848935486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4jx7t" (UID: "9efbb847-1884-4528-8390-64cb81e1f483") : secret "insights-runtime-extractor-tls" not found
Sep 15 12:45:36.056605 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:36.056590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efbb847-1884-4528-8390-64cb81e1f483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:37.317493 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:37.317452    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:37.317946 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:37.317530    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:37.317946 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:37.317887    2562 scope.go:117] "RemoveContainer" containerID="c9a6a4a4d0e8579feb182b304dd4fcdb9b3050d7cfd294333e2c6fb3da146600"
Sep 15 12:45:37.318096 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:37.318060    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-frn5n_openshift-console-operator(8d0130b6-3e38-41e4-a4dd-00d170040652)\"" pod="openshift-console-operator/console-operator-696884fb9c-frn5n" podUID="8d0130b6-3e38-41e4-a4dd-00d170040652"
Sep 15 12:45:41.927364 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:41.927334    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qpxx"
Sep 15 12:45:44.119540 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:44.119493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:44.121797 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:44.121773    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efbb847-1884-4528-8390-64cb81e1f483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jx7t\" (UID: \"9efbb847-1884-4528-8390-64cb81e1f483\") " pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:44.289666 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:44.289634    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmbfk\""
Sep 15 12:45:44.293442 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:44.293423    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4jx7t"
Sep 15 12:45:44.579415 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:44.579377    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4jx7t"]
Sep 15 12:45:44.582892 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:44.582857    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efbb847_1884_4528_8390_64cb81e1f483.slice/crio-f0333869faca5117a892bbd2c3f5067dd5a9494528cf2eed1eee82e8d8402e6d WatchSource:0}: Error finding container f0333869faca5117a892bbd2c3f5067dd5a9494528cf2eed1eee82e8d8402e6d: Status 404 returned error can't find the container with id f0333869faca5117a892bbd2c3f5067dd5a9494528cf2eed1eee82e8d8402e6d
Sep 15 12:45:45.056948 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:45.056913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jx7t" event={"ID":"9efbb847-1884-4528-8390-64cb81e1f483","Type":"ContainerStarted","Data":"46031d45a38927c1e27807bbb2f2d6fd9f5bd923557b52092885459014b9ba25"}
Sep 15 12:45:45.057127 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:45.056970    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jx7t" event={"ID":"9efbb847-1884-4528-8390-64cb81e1f483","Type":"ContainerStarted","Data":"f0333869faca5117a892bbd2c3f5067dd5a9494528cf2eed1eee82e8d8402e6d"}
Sep 15 12:45:46.065466 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:46.065423    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jx7t" event={"ID":"9efbb847-1884-4528-8390-64cb81e1f483","Type":"ContainerStarted","Data":"40d12a3aaceb13950e7c266066996a64d0a9afc1ed9b4b8e08acbcc338b03ff8"}
Sep 15 12:45:47.070826 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:47.070735    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jx7t" event={"ID":"9efbb847-1884-4528-8390-64cb81e1f483","Type":"ContainerStarted","Data":"66c1d62ae585ce9f0af7eb48c6e11d384f76aa8b8afdac210b3e403f88621713"}
Sep 15 12:45:47.115770 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:47.115726    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4jx7t" podStartSLOduration=17.090024182 podStartE2EDuration="19.11571128s" totalImagesPullingTime="2.025687098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:45:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:47.115240417 +0000 UTC m=+64.907942147" watchObservedRunningTime="2026-09-15 12:45:47.11571128 +0000 UTC m=+64.908413010"
Sep 15 12:45:48.457093 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.457050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:48.457539 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.457225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:48.460188 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.460167    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 12:45:48.461178 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.461160    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 12:45:48.470331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.470307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39d331a-f507-4475-9a80-16a20f6339b6-metrics-certs\") pod \"network-metrics-daemon-wdsdq\" (UID: \"a39d331a-f507-4475-9a80-16a20f6339b6\") " pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:48.470331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.470329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffa614d2-e094-43ee-a793-042735c1161d-original-pull-secret\") pod \"global-pull-secret-syncer-9wmfs\" (UID: \"ffa614d2-e094-43ee-a793-042735c1161d\") " pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:48.486304 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.486286    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w5cgx\""
Sep 15 12:45:48.494420 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.494402    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdsdq"
Sep 15 12:45:48.568902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.568344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:48.573781 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.573752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh8d5\" (UniqueName: \"kubernetes.io/projected/86965e08-cf54-4abf-a78a-b377d4e2d931-kube-api-access-bh8d5\") pod \"network-check-target-sm5v4\" (UID: \"86965e08-cf54-4abf-a78a-b377d4e2d931\") " pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:48.661631 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.661593    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wmfs"
Sep 15 12:45:48.779495 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.779470    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z9dbl\""
Sep 15 12:45:48.783417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.783391    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wdsdq"]
Sep 15 12:45:48.785020 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.784993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:48.786456 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:48.786433    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39d331a_f507_4475_9a80_16a20f6339b6.slice/crio-c60e7b2bfc091fd12700aa6481faad8be208fbaed4ef9398c74aca590598743e WatchSource:0}: Error finding container c60e7b2bfc091fd12700aa6481faad8be208fbaed4ef9398c74aca590598743e: Status 404 returned error can't find the container with id c60e7b2bfc091fd12700aa6481faad8be208fbaed4ef9398c74aca590598743e
Sep 15 12:45:48.870522 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.870483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:48.870522 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.870523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:48.870702 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.870550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:48.870738 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.870717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:48.870807 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.870786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:48.871440 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.871407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ee18f00-83f0-4177-906f-f66a46579c82-service-ca-bundle\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:48.872882 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.872858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5558222d-f542-4972-b7a1-a6b3425571cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b7j9r\" (UID: \"5558222d-f542-4972-b7a1-a6b3425571cd\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:48.872882 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.872879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"image-registry-788ffb67c9-gn56b\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") " pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:48.872882 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.872864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ee18f00-83f0-4177-906f-f66a46579c82-metrics-certs\") pod \"router-default-89fbc77c4-kzvnf\" (UID: \"7ee18f00-83f0-4177-906f-f66a46579c82\") " pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:48.873146 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.873081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/861581d3-d35a-46c8-85b3-72da2998195b-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-52749\" (UID: \"861581d3-d35a-46c8-85b3-72da2998195b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:48.953627 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.953598    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9wmfs"]
Sep 15 12:45:48.956874 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:48.956848    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa614d2_e094_43ee_a793_042735c1161d.slice/crio-b9c938a2c7e0fdb205f00cc230bf25f7d1f01e6106b16158a81235cd557be647 WatchSource:0}: Error finding container b9c938a2c7e0fdb205f00cc230bf25f7d1f01e6106b16158a81235cd557be647: Status 404 returned error can't find the container with id b9c938a2c7e0fdb205f00cc230bf25f7d1f01e6106b16158a81235cd557be647
Sep 15 12:45:48.971405 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.971385    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:48.973473 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:48.973454    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/436023f5-a121-4823-bf16-203409532181-metrics-tls\") pod \"dns-default-wnhvg\" (UID: \"436023f5-a121-4823-bf16-203409532181\") " pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:49.062885 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.062859    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sm5v4"]
Sep 15 12:45:49.065949 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:49.065916    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86965e08_cf54_4abf_a78a_b377d4e2d931.slice/crio-ef1c216efecb03ad6ae6cf36c93c50230b0ea4db02c8368be4165dfe271b96bd WatchSource:0}: Error finding container ef1c216efecb03ad6ae6cf36c93c50230b0ea4db02c8368be4165dfe271b96bd: Status 404 returned error can't find the container with id ef1c216efecb03ad6ae6cf36c93c50230b0ea4db02c8368be4165dfe271b96bd
Sep 15 12:45:49.077365 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.077343    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sm5v4" event={"ID":"86965e08-cf54-4abf-a78a-b377d4e2d931","Type":"ContainerStarted","Data":"ef1c216efecb03ad6ae6cf36c93c50230b0ea4db02c8368be4165dfe271b96bd"}
Sep 15 12:45:49.078399 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.078375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdsdq" event={"ID":"a39d331a-f507-4475-9a80-16a20f6339b6","Type":"ContainerStarted","Data":"c60e7b2bfc091fd12700aa6481faad8be208fbaed4ef9398c74aca590598743e"}
Sep 15 12:45:49.079491 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.079469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9wmfs" event={"ID":"ffa614d2-e094-43ee-a793-042735c1161d","Type":"ContainerStarted","Data":"b9c938a2c7e0fdb205f00cc230bf25f7d1f01e6106b16158a81235cd557be647"}
Sep 15 12:45:49.094948 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.094931    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ht9tr\""
Sep 15 12:45:49.101804 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.101785    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:49.109080 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.108972    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w89vc\""
Sep 15 12:45:49.112172 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.112127    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:49.115176 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.115151    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-88nkn\""
Sep 15 12:45:49.119163 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.119145    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"
Sep 15 12:45:49.127625 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.127602    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-w28t2\""
Sep 15 12:45:49.136300 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.136220    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"
Sep 15 12:45:49.174425 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.174014    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rtf69\""
Sep 15 12:45:49.181951 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.181142    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:49.412128 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.412091    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-788ffb67c9-gn56b"]
Sep 15 12:45:49.415556 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:49.415529    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1542bbc_a7c6_4873_8672_6607a20f7632.slice/crio-26dd957bc87e4fd56aeac958d4898e648fa01cf743ed64994460c32c6ea8389d WatchSource:0}: Error finding container 26dd957bc87e4fd56aeac958d4898e648fa01cf743ed64994460c32c6ea8389d: Status 404 returned error can't find the container with id 26dd957bc87e4fd56aeac958d4898e648fa01cf743ed64994460c32c6ea8389d
Sep 15 12:45:49.423297 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.423276    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r"]
Sep 15 12:45:49.429175 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:49.429141    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5558222d_f542_4972_b7a1_a6b3425571cd.slice/crio-e6a9358ff89a6b3441e1e002a1394a8fc26d39395d9578975f1882c3a475951a WatchSource:0}: Error finding container e6a9358ff89a6b3441e1e002a1394a8fc26d39395d9578975f1882c3a475951a: Status 404 returned error can't find the container with id e6a9358ff89a6b3441e1e002a1394a8fc26d39395d9578975f1882c3a475951a
Sep 15 12:45:49.441207 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.441181    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-89fbc77c4-kzvnf"]
Sep 15 12:45:49.445686 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.445606    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749"]
Sep 15 12:45:49.447885 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:49.447815    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee18f00_83f0_4177_906f_f66a46579c82.slice/crio-e883ce72b56dfaffe681f5ebcff8578b034e1352ad91f5c95c9c069a3b24d830 WatchSource:0}: Error finding container e883ce72b56dfaffe681f5ebcff8578b034e1352ad91f5c95c9c069a3b24d830: Status 404 returned error can't find the container with id e883ce72b56dfaffe681f5ebcff8578b034e1352ad91f5c95c9c069a3b24d830
Sep 15 12:45:49.500557 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:49.500520    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wnhvg"]
Sep 15 12:45:49.771578 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:45:49.771504    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436023f5_a121_4823_bf16_203409532181.slice/crio-3e71fccb9b8dbf7f732031427d317b9936a0ac4b8c478aaeec0292fb6aa6aa61 WatchSource:0}: Error finding container 3e71fccb9b8dbf7f732031427d317b9936a0ac4b8c478aaeec0292fb6aa6aa61: Status 404 returned error can't find the container with id 3e71fccb9b8dbf7f732031427d317b9936a0ac4b8c478aaeec0292fb6aa6aa61
Sep 15 12:45:50.085997 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.085916    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wnhvg" event={"ID":"436023f5-a121-4823-bf16-203409532181","Type":"ContainerStarted","Data":"3e71fccb9b8dbf7f732031427d317b9936a0ac4b8c478aaeec0292fb6aa6aa61"}
Sep 15 12:45:50.087440 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.087403    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749" event={"ID":"861581d3-d35a-46c8-85b3-72da2998195b","Type":"ContainerStarted","Data":"81d2ad641f29dc419a94ac07fa904adf84731734b1ef4c60a8ababe47a1e227f"}
Sep 15 12:45:50.088837 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.088804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r" event={"ID":"5558222d-f542-4972-b7a1-a6b3425571cd","Type":"ContainerStarted","Data":"e6a9358ff89a6b3441e1e002a1394a8fc26d39395d9578975f1882c3a475951a"}
Sep 15 12:45:50.090708 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.090679    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sm5v4" event={"ID":"86965e08-cf54-4abf-a78a-b377d4e2d931","Type":"ContainerStarted","Data":"f799f7c4faf141769591ff9117c60f2659a458e83091b00223541ac2d523ee72"}
Sep 15 12:45:50.090850 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.090789    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:45:50.093195 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.093172    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdsdq" event={"ID":"a39d331a-f507-4475-9a80-16a20f6339b6","Type":"ContainerStarted","Data":"e8e137348dbaa808b3992c9137bf592703baf343515e1c1f0ee8ce55767210c1"}
Sep 15 12:45:50.093303 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.093217    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdsdq" event={"ID":"a39d331a-f507-4475-9a80-16a20f6339b6","Type":"ContainerStarted","Data":"87a58e05032cdeab78b70a203db3034737ea906985d91962b0e2ddaae0ab5301"}
Sep 15 12:45:50.095485 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.095325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-89fbc77c4-kzvnf" event={"ID":"7ee18f00-83f0-4177-906f-f66a46579c82","Type":"ContainerStarted","Data":"fcc516331d89b48bc7650e6f83ded19272a855026dbba0be8f516bb76ac69f10"}
Sep 15 12:45:50.095485 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.095370    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-89fbc77c4-kzvnf" event={"ID":"7ee18f00-83f0-4177-906f-f66a46579c82","Type":"ContainerStarted","Data":"e883ce72b56dfaffe681f5ebcff8578b034e1352ad91f5c95c9c069a3b24d830"}
Sep 15 12:45:50.097322 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.097298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b" event={"ID":"c1542bbc-a7c6-4873-8672-6607a20f7632","Type":"ContainerStarted","Data":"4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744"}
Sep 15 12:45:50.097410 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.097338    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b" event={"ID":"c1542bbc-a7c6-4873-8672-6607a20f7632","Type":"ContainerStarted","Data":"26dd957bc87e4fd56aeac958d4898e648fa01cf743ed64994460c32c6ea8389d"}
Sep 15 12:45:50.097479 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.097431    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:45:50.112966 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.112922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:50.112966 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.112912    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sm5v4" podStartSLOduration=67.112895173 podStartE2EDuration="1m7.112895173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:50.111758719 +0000 UTC m=+67.904460451" watchObservedRunningTime="2026-09-15 12:45:50.112895173 +0000 UTC m=+67.905596906"
Sep 15 12:45:50.115702 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.115683    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:50.145251 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.145202    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b" podStartSLOduration=67.145187185 podStartE2EDuration="1m7.145187185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:50.144911015 +0000 UTC m=+67.937612745" watchObservedRunningTime="2026-09-15 12:45:50.145187185 +0000 UTC m=+67.937888917"
Sep 15 12:45:50.183589 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.183532    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-89fbc77c4-kzvnf" podStartSLOduration=55.183516988 podStartE2EDuration="55.183516988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:50.183002089 +0000 UTC m=+67.975703821" watchObservedRunningTime="2026-09-15 12:45:50.183516988 +0000 UTC m=+67.976218719"
Sep 15 12:45:50.215313 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:50.215248    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wdsdq" podStartSLOduration=66.206887483 podStartE2EDuration="1m7.215225326s" totalImagesPullingTime="1.008337843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:50.213325959 +0000 UTC m=+68.006027691" watchObservedRunningTime="2026-09-15 12:45:50.215225326 +0000 UTC m=+68.007927058"
Sep 15 12:45:51.103368 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:51.103192    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:51.104643 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:51.104615    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-89fbc77c4-kzvnf"
Sep 15 12:45:51.750738 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:51.750705    2562 scope.go:117] "RemoveContainer" containerID="c9a6a4a4d0e8579feb182b304dd4fcdb9b3050d7cfd294333e2c6fb3da146600"
Sep 15 12:45:54.119886 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.119812    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:45:54.120337 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.119920    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-frn5n" event={"ID":"8d0130b6-3e38-41e4-a4dd-00d170040652","Type":"ContainerStarted","Data":"1f1837d59e66c3f1fadfef991e27010e235cfe3e637c5b8087b521e8f0b10a7b"}
Sep 15 12:45:54.120626 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.120416    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:54.122141 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.122119    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9wmfs" event={"ID":"ffa614d2-e094-43ee-a793-042735c1161d","Type":"ContainerStarted","Data":"8dbf5f6cf4b3283b74543268761a4e7f03688415a8728d94d5eb94fe33bc0945"}
Sep 15 12:45:54.123789 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.123769    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749" event={"ID":"861581d3-d35a-46c8-85b3-72da2998195b","Type":"ContainerStarted","Data":"297673e3c9d8897c20c91014706e726fa60e5be70d433fbcc1de924acdb5bdf8"}
Sep 15 12:45:54.125419 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.125385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r" event={"ID":"5558222d-f542-4972-b7a1-a6b3425571cd","Type":"ContainerStarted","Data":"fb8b83d8cbd82cdf5d8ddee64b991f83fbe2c1fd08933254c2e186ff5b3d3450"}
Sep 15 12:45:54.158594 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.158550    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-frn5n" podStartSLOduration=49.977669631 podStartE2EDuration="59.158535716s" totalImagesPullingTime="9.180866085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:54.158184448 +0000 UTC m=+71.950886180" watchObservedRunningTime="2026-09-15 12:45:54.158535716 +0000 UTC m=+71.951237446"
Sep 15 12:45:54.213825 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.213787    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-frn5n"
Sep 15 12:45:54.240260 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.239848    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b7j9r" podStartSLOduration=54.887109299 podStartE2EDuration="59.239826372s" totalImagesPullingTime="4.352717073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:54.238686163 +0000 UTC m=+72.031387895" watchObservedRunningTime="2026-09-15 12:45:54.239826372 +0000 UTC m=+72.032528122"
Sep 15 12:45:54.312924 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:54.312864    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9wmfs" podStartSLOduration=66.487172449 podStartE2EDuration="1m11.312842492s" totalImagesPullingTime="4.825670043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:54.312688704 +0000 UTC m=+72.105390436" watchObservedRunningTime="2026-09-15 12:45:54.312842492 +0000 UTC m=+72.105544226"
Sep 15 12:45:55.131658 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:55.131619    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wnhvg" event={"ID":"436023f5-a121-4823-bf16-203409532181","Type":"ContainerStarted","Data":"f5617043ee550049491e2f3564b415b8905fe5b0dda9a9e76b2173ad793d3fd2"}
Sep 15 12:45:55.132082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:55.131681    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wnhvg" event={"ID":"436023f5-a121-4823-bf16-203409532181","Type":"ContainerStarted","Data":"911642493ea460fb5ba990c4605be1966298819e8eb4becd35109708fcc60411"}
Sep 15 12:45:55.132082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:55.131753    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:45:55.133633 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:55.133607    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749" event={"ID":"861581d3-d35a-46c8-85b3-72da2998195b","Type":"ContainerStarted","Data":"5a905404e0ac1e6243e03d759ef375d9bb07d90c39c2c9d21def5af3debcf3ee"}
Sep 15 12:45:55.470493 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:55.470442    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wnhvg" podStartSLOduration=35.087907839 podStartE2EDuration="39.470423238s" totalImagesPullingTime="4.382515399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:55.257764185 +0000 UTC m=+73.050465916" watchObservedRunningTime="2026-09-15 12:45:55.470423238 +0000 UTC m=+73.263124976"
Sep 15 12:45:58.448652 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.448597    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-52749" podStartSLOduration=59.202937884 podStartE2EDuration="1m3.448579353s" totalImagesPullingTime="4.245641469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:45:55.470392848 +0000 UTC m=+73.263094579" watchObservedRunningTime="2026-09-15 12:45:58.448579353 +0000 UTC m=+76.241281131"
Sep 15 12:45:58.449410 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.449391    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"]
Sep 15 12:45:58.484508 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.484481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"]
Sep 15 12:45:58.484646 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.484596    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.493269 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.493247    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 15 12:45:58.493384 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.493318    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 12:45:58.493446 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.493376    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 15 12:45:58.494343 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.494327    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bwm4p\""
Sep 15 12:45:58.651592 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.651557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a18ffd30-7494-4446-8266-df7dfb5f9365-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.651756 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.651601    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6747t\" (UniqueName: \"kubernetes.io/projected/a18ffd30-7494-4446-8266-df7dfb5f9365-kube-api-access-6747t\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.651756 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.651623    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.651756 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.651692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.752373 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.752291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.752373 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.752347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a18ffd30-7494-4446-8266-df7dfb5f9365-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.752595 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.752395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6747t\" (UniqueName: \"kubernetes.io/projected/a18ffd30-7494-4446-8266-df7dfb5f9365-kube-api-access-6747t\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.752595 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.752425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.752595 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:58.752531    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 15 12:45:58.752735 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:58.752601    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls podName:a18ffd30-7494-4446-8266-df7dfb5f9365 nodeName:}" failed. No retries permitted until 2026-09-15 12:45:59.252580097 +0000 UTC m=+77.045281809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-p76dz" (UID: "a18ffd30-7494-4446-8266-df7dfb5f9365") : secret "prometheus-operator-tls" not found
Sep 15 12:45:58.753120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.753093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a18ffd30-7494-4446-8266-df7dfb5f9365-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.755159 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.755136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:58.771230 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:58.771212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6747t\" (UniqueName: \"kubernetes.io/projected/a18ffd30-7494-4446-8266-df7dfb5f9365-kube-api-access-6747t\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:59.256715 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:45:59.256678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:45:59.256884 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:59.256804    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 15 12:45:59.256884 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:45:59.256867    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls podName:a18ffd30-7494-4446-8266-df7dfb5f9365 nodeName:}" failed. No retries permitted until 2026-09-15 12:46:00.25684383 +0000 UTC m=+78.049545541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-p76dz" (UID: "a18ffd30-7494-4446-8266-df7dfb5f9365") : secret "prometheus-operator-tls" not found
Sep 15 12:46:00.265961 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:00.265918    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:46:00.268292 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:00.268272    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a18ffd30-7494-4446-8266-df7dfb5f9365-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-p76dz\" (UID: \"a18ffd30-7494-4446-8266-df7dfb5f9365\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:46:00.293314 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:00.293281    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"
Sep 15 12:46:00.594292 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:00.594214    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-p76dz"]
Sep 15 12:46:00.597055 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:46:00.597013    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18ffd30_7494_4446_8266_df7dfb5f9365.slice/crio-4a434d0b586c1a89936ab5cb3056c82a7dcc378dc89e7e9b1f18c8fb76dfe37b WatchSource:0}: Error finding container 4a434d0b586c1a89936ab5cb3056c82a7dcc378dc89e7e9b1f18c8fb76dfe37b: Status 404 returned error can't find the container with id 4a434d0b586c1a89936ab5cb3056c82a7dcc378dc89e7e9b1f18c8fb76dfe37b
Sep 15 12:46:01.162825 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:01.162787    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz" event={"ID":"a18ffd30-7494-4446-8266-df7dfb5f9365","Type":"ContainerStarted","Data":"4a434d0b586c1a89936ab5cb3056c82a7dcc378dc89e7e9b1f18c8fb76dfe37b"}
Sep 15 12:46:03.171986 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:03.171952    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz" event={"ID":"a18ffd30-7494-4446-8266-df7dfb5f9365","Type":"ContainerStarted","Data":"361051b9397ee15a4f5202d6c360767d17723885168f216ba357f9b6f28bc002"}
Sep 15 12:46:03.172574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:03.172011    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz" event={"ID":"a18ffd30-7494-4446-8266-df7dfb5f9365","Type":"ContainerStarted","Data":"47cc14886c59850ca57da01719ebda57604916556185647a6c8ff46269b10adc"}
Sep 15 12:46:03.198523 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:03.198475    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-p76dz" podStartSLOduration=3.5314646119999997 podStartE2EDuration="5.198459101s" totalImagesPullingTime="1.666994489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:45:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:03.197329276 +0000 UTC m=+80.990031007" watchObservedRunningTime="2026-09-15 12:46:03.198459101 +0000 UTC m=+80.991160831"
Sep 15 12:46:05.140060 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.139932    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wnhvg"
Sep 15 12:46:05.205548 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.205514    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"]
Sep 15 12:46:05.211089 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.208607    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wzvww"]
Sep 15 12:46:05.211600 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.211580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.212239 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.212218    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.220995 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:05.220970    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-dockercfg-v6rm8\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-v6rm8\"" type="*v1.Secret"
Sep 15 12:46:05.221738 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:05.221702    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" type="*v1.Secret"
Sep 15 12:46:05.221831 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:05.221756    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-tls\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" type="*v1.Secret"
Sep 15 12:46:05.221831 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:05.221755    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-state-metrics-custom-resource-state-configmap\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" type="*v1.ConfigMap"
Sep 15 12:46:05.221831 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:05.221757    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap"
Sep 15 12:46:05.222297 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:05.222271    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-9phj7\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9phj7\"" type="*v1.Secret"
Sep 15 12:46:05.222833 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:05.222815    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret"
Sep 15 12:46:05.227462 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:05.227438    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret"
Sep 15 12:46:05.257481 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.257453    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"]
Sep 15 12:46:05.306068 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306024    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-root\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.306216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306103    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37cdf835-8faf-4180-83e4-40b96a85ca03-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.306216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-tls\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.306216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-wtmp\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.306216 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-metrics-client-ca\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.306350 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37cdf835-8faf-4180-83e4-40b96a85ca03-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.306350 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.306350 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dktfj\" (UniqueName: \"kubernetes.io/projected/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-kube-api-access-dktfj\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.306350 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.306350 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-textfile\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.306565 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.306565 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.306565 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-sys\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.306565 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306473    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkc5j\" (UniqueName: \"kubernetes.io/projected/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-api-access-vkc5j\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.306565 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.306503    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-accelerators-collector-config\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.407615 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-metrics-client-ca\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.407615 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37cdf835-8faf-4180-83e4-40b96a85ca03-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.407615 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407593    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.407615 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dktfj\" (UniqueName: \"kubernetes.io/projected/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-kube-api-access-dktfj\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.407920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.407920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407641    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-textfile\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.407920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.407920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.407920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-sys\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.407920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407909    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkc5j\" (UniqueName: \"kubernetes.io/projected/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-api-access-vkc5j\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-accelerators-collector-config\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.407957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-sys\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408002    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37cdf835-8faf-4180-83e4-40b96a85ca03-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-root\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408056    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-root\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37cdf835-8faf-4180-83e4-40b96a85ca03-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-tls\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-wtmp\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408242 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-textfile\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408516 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-metrics-client-ca\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408516 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-wtmp\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.408617 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.408599    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37cdf835-8faf-4180-83e4-40b96a85ca03-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:05.434315 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.434283    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dktfj\" (UniqueName: \"kubernetes.io/projected/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-kube-api-access-dktfj\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:05.452480 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:05.452448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkc5j\" (UniqueName: \"kubernetes.io/projected/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-api-access-vkc5j\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:06.263325 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.263292    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 15 12:46:06.268853 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.268826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:06.279009 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.278983    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-v6rm8\""
Sep 15 12:46:06.349759 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.349730    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 15 12:46:06.360578 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.360546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:06.407899 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:06.407871    2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: failed to sync secret cache: timed out waiting for the condition
Sep 15 12:46:06.408056 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:06.407946    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-tls podName:37cdf835-8faf-4180-83e4-40b96a85ca03 nodeName:}" failed. No retries permitted until 2026-09-15 12:46:06.907930167 +0000 UTC m=+84.700631881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-jqcbj" (UID: "37cdf835-8faf-4180-83e4-40b96a85ca03") : failed to sync secret cache: timed out waiting for the condition
Sep 15 12:46:06.408056 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:06.407871    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-kube-rbac-proxy-config: failed to sync secret cache: timed out waiting for the condition
Sep 15 12:46:06.408056 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:06.408012    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-kube-rbac-proxy-config podName:7ff0210f-ee7a-4eb9-bd17-c8396da63af6 nodeName:}" failed. No retries permitted until 2026-09-15 12:46:06.907998001 +0000 UTC m=+84.700699723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-kube-rbac-proxy-config" (UniqueName: "kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-kube-rbac-proxy-config") pod "node-exporter-wzvww" (UID: "7ff0210f-ee7a-4eb9-bd17-c8396da63af6") : failed to sync secret cache: timed out waiting for the condition
Sep 15 12:46:06.408056 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:06.408052    2562 configmap.go:193] Couldn't get configMap openshift-monitoring/node-exporter-accelerators-collector-config: failed to sync configmap cache: timed out waiting for the condition
Sep 15 12:46:06.408223 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:06.408090    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-accelerators-collector-config podName:7ff0210f-ee7a-4eb9-bd17-c8396da63af6 nodeName:}" failed. No retries permitted until 2026-09-15 12:46:06.908081124 +0000 UTC m=+84.700782847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-accelerators-collector-config" (UniqueName: "kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-accelerators-collector-config") pod "node-exporter-wzvww" (UID: "7ff0210f-ee7a-4eb9-bd17-c8396da63af6") : failed to sync configmap cache: timed out waiting for the condition
Sep 15 12:46:06.408951 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:06.408932    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: failed to sync secret cache: timed out waiting for the condition
Sep 15 12:46:06.408999 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:06.408994    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-tls podName:7ff0210f-ee7a-4eb9-bd17-c8396da63af6 nodeName:}" failed. No retries permitted until 2026-09-15 12:46:06.908980126 +0000 UTC m=+84.701681847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-tls") pod "node-exporter-wzvww" (UID: "7ff0210f-ee7a-4eb9-bd17-c8396da63af6") : failed to sync secret cache: timed out waiting for the condition
Sep 15 12:46:06.487655 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.487625    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9phj7\""
Sep 15 12:46:06.557433 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.557360    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 12:46:06.642674 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.642643    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 12:46:06.686224 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.686194    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 15 12:46:06.820232 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.820156    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 12:46:06.922202 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.922166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-tls\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:06.922363 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.922231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:06.922363 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.922281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:06.922462 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.922372    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-accelerators-collector-config\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:06.922855 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.922833    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-accelerators-collector-config\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:06.924698 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.924669    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-tls\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:06.925119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.925095    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ff0210f-ee7a-4eb9-bd17-c8396da63af6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wzvww\" (UID: \"7ff0210f-ee7a-4eb9-bd17-c8396da63af6\") " pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:06.925220 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:06.925207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37cdf835-8faf-4180-83e4-40b96a85ca03-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-jqcbj\" (UID: \"37cdf835-8faf-4180-83e4-40b96a85ca03\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:07.021273 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:07.021231    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"
Sep 15 12:46:07.026886 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:07.026862    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wzvww"
Sep 15 12:46:07.038298 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:46:07.038266    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff0210f_ee7a_4eb9_bd17_c8396da63af6.slice/crio-d7e40e8d066707a1e543b87891c16f38bb72feff69421c0bcb0fcffd82d87cf0 WatchSource:0}: Error finding container d7e40e8d066707a1e543b87891c16f38bb72feff69421c0bcb0fcffd82d87cf0: Status 404 returned error can't find the container with id d7e40e8d066707a1e543b87891c16f38bb72feff69421c0bcb0fcffd82d87cf0
Sep 15 12:46:07.188026 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:07.187991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wzvww" event={"ID":"7ff0210f-ee7a-4eb9-bd17-c8396da63af6","Type":"ContainerStarted","Data":"d7e40e8d066707a1e543b87891c16f38bb72feff69421c0bcb0fcffd82d87cf0"}
Sep 15 12:46:07.332413 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:07.332386    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj"]
Sep 15 12:46:07.334386 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:46:07.334358    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37cdf835_8faf_4180_83e4_40b96a85ca03.slice/crio-b5fc1996a228758befcdabe918c54747a32d4668ba6343f214d3b3502c507028 WatchSource:0}: Error finding container b5fc1996a228758befcdabe918c54747a32d4668ba6343f214d3b3502c507028: Status 404 returned error can't find the container with id b5fc1996a228758befcdabe918c54747a32d4668ba6343f214d3b3502c507028
Sep 15 12:46:08.194831 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.194755    2562 generic.go:356] "Generic (PLEG): container finished" podID="7ff0210f-ee7a-4eb9-bd17-c8396da63af6" containerID="12547c1313942be63f241da22fcd7fa1374e2fc3c873e2a7330ab856572185fa" exitCode=0
Sep 15 12:46:08.194985 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.194852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wzvww" event={"ID":"7ff0210f-ee7a-4eb9-bd17-c8396da63af6","Type":"ContainerDied","Data":"12547c1313942be63f241da22fcd7fa1374e2fc3c873e2a7330ab856572185fa"}
Sep 15 12:46:08.196328 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.196286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj" event={"ID":"37cdf835-8faf-4180-83e4-40b96a85ca03","Type":"ContainerStarted","Data":"b5fc1996a228758befcdabe918c54747a32d4668ba6343f214d3b3502c507028"}
Sep 15 12:46:08.247401 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.247371    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-cccf48976-k6cbj"]
Sep 15 12:46:08.250445 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.250424    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.256819 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.256797    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 15 12:46:08.256942 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.256797    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 15 12:46:08.259803 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.259781    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 15 12:46:08.260149 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.260126    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 15 12:46:08.260294 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.260271    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bpnqk\""
Sep 15 12:46:08.260502 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.260485    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4531ee2g7felu\""
Sep 15 12:46:08.261616 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.261597    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 15 12:46:08.274045 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.274011    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-cccf48976-k6cbj"]
Sep 15 12:46:08.435299 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.435260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.435299 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.435303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a425049-8fbb-4c61-9017-b9e410487d0a-metrics-client-ca\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.435764 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.435353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-tls\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.435764 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.435383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.435764 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.435418    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.435764 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.435441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-grpc-tls\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.435764 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.435483    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njpnb\" (UniqueName: \"kubernetes.io/projected/4a425049-8fbb-4c61-9017-b9e410487d0a-kube-api-access-njpnb\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.435764 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.435540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.535938 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.535904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.536103 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.535951    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a425049-8fbb-4c61-9017-b9e410487d0a-metrics-client-ca\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.536103 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.535992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-tls\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.536103 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.536018    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.536103 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.536069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.536103 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.536091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-grpc-tls\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.536418 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.536126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njpnb\" (UniqueName: \"kubernetes.io/projected/4a425049-8fbb-4c61-9017-b9e410487d0a-kube-api-access-njpnb\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.536418 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.536153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.536907 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.536881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a425049-8fbb-4c61-9017-b9e410487d0a-metrics-client-ca\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.539505 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.539478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-tls\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.540006 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.539861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-grpc-tls\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.540456 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.540171    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.540859 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.540834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.540955 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.540866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.541952 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.541931    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a425049-8fbb-4c61-9017-b9e410487d0a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.550073 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.550022    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njpnb\" (UniqueName: \"kubernetes.io/projected/4a425049-8fbb-4c61-9017-b9e410487d0a-kube-api-access-njpnb\") pod \"thanos-querier-cccf48976-k6cbj\" (UID: \"4a425049-8fbb-4c61-9017-b9e410487d0a\") " pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.561534 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.561512    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:08.881781 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:08.881749    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-cccf48976-k6cbj"]
Sep 15 12:46:08.882326 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:46:08.882296    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a425049_8fbb_4c61_9017_b9e410487d0a.slice/crio-78106727eb8c2acec187d7927d7a148d8ca83aaf14818770c6316e5839fb0227 WatchSource:0}: Error finding container 78106727eb8c2acec187d7927d7a148d8ca83aaf14818770c6316e5839fb0227: Status 404 returned error can't find the container with id 78106727eb8c2acec187d7927d7a148d8ca83aaf14818770c6316e5839fb0227
Sep 15 12:46:09.201787 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.201749    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj" event={"ID":"4a425049-8fbb-4c61-9017-b9e410487d0a","Type":"ContainerStarted","Data":"78106727eb8c2acec187d7927d7a148d8ca83aaf14818770c6316e5839fb0227"}
Sep 15 12:46:09.203916 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.203890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wzvww" event={"ID":"7ff0210f-ee7a-4eb9-bd17-c8396da63af6","Type":"ContainerStarted","Data":"28d7c86567f4ba222e790d4e6876426912646cf0f071647169290c9767a0c8db"}
Sep 15 12:46:09.204054 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.203939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wzvww" event={"ID":"7ff0210f-ee7a-4eb9-bd17-c8396da63af6","Type":"ContainerStarted","Data":"cd084695d1246549a0bbefbee0dc5c2f80bcbb12ad4b4fb1b8996208e4924933"}
Sep 15 12:46:09.206140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.206116    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj" event={"ID":"37cdf835-8faf-4180-83e4-40b96a85ca03","Type":"ContainerStarted","Data":"c0eac41286bd0ecbc984f86c6f0066a75c75ca7359d9d43b8ed56fd536cc2ca5"}
Sep 15 12:46:09.206235 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.206160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj" event={"ID":"37cdf835-8faf-4180-83e4-40b96a85ca03","Type":"ContainerStarted","Data":"ba5fe91d093a72cb40f5d804d34959b0e836a96a540e3e2d4daa486b95174c9a"}
Sep 15 12:46:09.206235 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.206183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj" event={"ID":"37cdf835-8faf-4180-83e4-40b96a85ca03","Type":"ContainerStarted","Data":"52a3d45f931b07e323820e69534bae393cc3b73aba31b529f0f1d9e2f1371fab"}
Sep 15 12:46:09.258746 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.258689    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wzvww" podStartSLOduration=3.286295342 podStartE2EDuration="4.258674441s" totalImagesPullingTime="897.258036ms" totalInitContainerRuntime="75.121063ms" isStatefulPod=true podCreationTimestamp="2026-09-15 12:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:09.256950702 +0000 UTC m=+87.049652435" watchObservedRunningTime="2026-09-15 12:46:09.258674441 +0000 UTC m=+87.051376201"
Sep 15 12:46:09.985227 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.985155    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-jqcbj" podStartSLOduration=3.833457389 podStartE2EDuration="4.985129872s" totalImagesPullingTime="1.151672483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:09.312077832 +0000 UTC m=+87.104779563" watchObservedRunningTime="2026-09-15 12:46:09.985129872 +0000 UTC m=+87.777831605"
Sep 15 12:46:09.986917 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.986892    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-784857c55d-j5dnw"]
Sep 15 12:46:09.989292 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:09.989273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.006415 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.006392    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 12:46:10.006560 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.006541    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 15 12:46:10.006690 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.006667    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 15 12:46:10.007393 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.007368    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 15 12:46:10.007487 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.007468    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4qwt8\""
Sep 15 12:46:10.007589 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.007575    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4nras4fk1etmu\""
Sep 15 12:46:10.019145 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.019112    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-784857c55d-j5dnw"]
Sep 15 12:46:10.150283 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.150245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-secret-metrics-server-tls\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.150462 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.150313    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a800ca98-8cb1-458d-b95d-0af6ac38e349-audit-log\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.150462 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.150368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26j6\" (UniqueName: \"kubernetes.io/projected/a800ca98-8cb1-458d-b95d-0af6ac38e349-kube-api-access-z26j6\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.150462 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.150408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-secret-metrics-server-client-certs\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.150462 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.150433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-client-ca-bundle\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.150462 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.150461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a800ca98-8cb1-458d-b95d-0af6ac38e349-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.150727 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.150508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a800ca98-8cb1-458d-b95d-0af6ac38e349-metrics-server-audit-profiles\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251106 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a800ca98-8cb1-458d-b95d-0af6ac38e349-metrics-server-audit-profiles\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251269 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251112    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-secret-metrics-server-tls\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251269 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a800ca98-8cb1-458d-b95d-0af6ac38e349-audit-log\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251269 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26j6\" (UniqueName: \"kubernetes.io/projected/a800ca98-8cb1-458d-b95d-0af6ac38e349-kube-api-access-z26j6\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251269 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251238    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-secret-metrics-server-client-certs\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251269 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-client-ca-bundle\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251519 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251286    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a800ca98-8cb1-458d-b95d-0af6ac38e349-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251618 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a800ca98-8cb1-458d-b95d-0af6ac38e349-audit-log\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.251968 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.251929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a800ca98-8cb1-458d-b95d-0af6ac38e349-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.252112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.252094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a800ca98-8cb1-458d-b95d-0af6ac38e349-metrics-server-audit-profiles\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.253594 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.253563    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-secret-metrics-server-client-certs\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.253695 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.253604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-client-ca-bundle\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.253695 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.253674    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a800ca98-8cb1-458d-b95d-0af6ac38e349-secret-metrics-server-tls\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.264342 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.264318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26j6\" (UniqueName: \"kubernetes.io/projected/a800ca98-8cb1-458d-b95d-0af6ac38e349-kube-api-access-z26j6\") pod \"metrics-server-784857c55d-j5dnw\" (UID: \"a800ca98-8cb1-458d-b95d-0af6ac38e349\") " pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.300976 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.300949    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:10.592678 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:10.592579    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-784857c55d-j5dnw"]
Sep 15 12:46:10.908336 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:46:10.908275    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda800ca98_8cb1_458d_b95d_0af6ac38e349.slice/crio-dbf16441725d44c134b446eb7c637243ca36f2421fa2396f21d2deef2f67383b WatchSource:0}: Error finding container dbf16441725d44c134b446eb7c637243ca36f2421fa2396f21d2deef2f67383b: Status 404 returned error can't find the container with id dbf16441725d44c134b446eb7c637243ca36f2421fa2396f21d2deef2f67383b
Sep 15 12:46:11.107782 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.107754    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:46:11.216866 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.216833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj" event={"ID":"4a425049-8fbb-4c61-9017-b9e410487d0a","Type":"ContainerStarted","Data":"18e701f3c402a9ce5d4df29d7548271d1b506b21542e67b594ae6eb207281be1"}
Sep 15 12:46:11.217052 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.216910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj" event={"ID":"4a425049-8fbb-4c61-9017-b9e410487d0a","Type":"ContainerStarted","Data":"944463c56139fcb4aaae58c0d147d2d6e4395645473010a2ef5bb2f0d0116bf6"}
Sep 15 12:46:11.217052 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.216952    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj" event={"ID":"4a425049-8fbb-4c61-9017-b9e410487d0a","Type":"ContainerStarted","Data":"b553901b8c11f51981be1c7b2153b708b3f501a60da72b010381f3af193ae3c0"}
Sep 15 12:46:11.218014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.217991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw" event={"ID":"a800ca98-8cb1-458d-b95d-0af6ac38e349","Type":"ContainerStarted","Data":"dbf16441725d44c134b446eb7c637243ca36f2421fa2396f21d2deef2f67383b"}
Sep 15 12:46:11.795499 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.795447    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:46:11.798506 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.798484    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.803797 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.803078    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 12:46:11.803797 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.803403    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 12:46:11.803797 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.803637    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 12:46:11.804177 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.804156    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 12:46:11.804379 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.804256    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 12:46:11.804379 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.804310    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 12:46:11.804545 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.804439    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 12:46:11.804545 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.804311    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 12:46:11.804649 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.804636    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l8z4s\""
Sep 15 12:46:11.804905 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.804744    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 12:46:11.805853 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.805829    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f7mid7225h89a\""
Sep 15 12:46:11.805950 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.805911    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 12:46:11.806009 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.805943    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 12:46:11.806098 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.806022    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 12:46:11.806750 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.806574    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 12:46:11.807318 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.807120    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 12:46:11.824135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.824112    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:46:11.866908 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.866871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.866919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.866994    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867024    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-config\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867113    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-config-out\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867160    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867727 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867727 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bw7w\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-kube-api-access-4bw7w\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867727 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867449    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867727 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867479    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-web-config\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867727 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.867727 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.867534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.967928 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.967894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.967937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.967960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.967989    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bw7w\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-kube-api-access-4bw7w\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968172    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-web-config\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-config\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968417    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968966 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968966 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-config-out\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968966 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968524    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.968966 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.968866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.970257 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.969460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.970257 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.969837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.970257 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.969906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.971044 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.970724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.971044 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.971001    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.971747 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.971715    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.973513 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.973488    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.976993 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.975370    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.976993 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.976141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.976993 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.976596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-config\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.977199 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.977171    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.977299 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.977257    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.977299 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.977260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.977635 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.977610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.977748 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.977656    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.977748 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.977695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.978025 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.978006    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-web-config\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.978125 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.978066    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-config-out\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:11.981689 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:11.981635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bw7w\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-kube-api-access-4bw7w\") pod \"prometheus-k8s-0\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:12.115924 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:12.115899    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:12.228364 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:12.228329    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj" event={"ID":"4a425049-8fbb-4c61-9017-b9e410487d0a","Type":"ContainerStarted","Data":"2fe6d360fc231a366f50dd60f1b249235989ba87330c5c90283b96a7f96f9931"}
Sep 15 12:46:12.228483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:12.228449    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj" event={"ID":"4a425049-8fbb-4c61-9017-b9e410487d0a","Type":"ContainerStarted","Data":"9b3abda53d76c89fe4be8c01f077a7c4be134843b7a29be74d29e8c5eea8cfee"}
Sep 15 12:46:12.228525 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:12.228490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj" event={"ID":"4a425049-8fbb-4c61-9017-b9e410487d0a","Type":"ContainerStarted","Data":"b3f92a2af3f459f62673fc47aaf5c0c8aea8a133db33128c53b86cf66a2ae473"}
Sep 15 12:46:12.228559 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:12.228531    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:12.262104 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:12.262056    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj" podStartSLOduration=1.293800739 podStartE2EDuration="4.262018377s" totalImagesPullingTime="2.968217638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:46:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:12.259684547 +0000 UTC m=+90.052386279" watchObservedRunningTime="2026-09-15 12:46:12.262018377 +0000 UTC m=+90.054720108"
Sep 15 12:46:12.407834 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:12.407602    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:46:12.410242 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:46:12.410216    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390c18d0_e350_4936_b3fd_893e4ec69107.slice/crio-0c08be2bca331998956bbe990b4b2d66fdb174f90f56be8c465e90d1d4baea7b WatchSource:0}: Error finding container 0c08be2bca331998956bbe990b4b2d66fdb174f90f56be8c465e90d1d4baea7b: Status 404 returned error can't find the container with id 0c08be2bca331998956bbe990b4b2d66fdb174f90f56be8c465e90d1d4baea7b
Sep 15 12:46:13.234266 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:13.234229    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerStarted","Data":"0c08be2bca331998956bbe990b4b2d66fdb174f90f56be8c465e90d1d4baea7b"}
Sep 15 12:46:14.240208 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:14.240171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw" event={"ID":"a800ca98-8cb1-458d-b95d-0af6ac38e349","Type":"ContainerStarted","Data":"2b9e6f3637bd7a04dc425e49f77cd7633723a72f28ec6b3b8fa5e894d0a41fa2"}
Sep 15 12:46:14.241693 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:14.241667    2562 generic.go:356] "Generic (PLEG): container finished" podID="390c18d0-e350-4936-b3fd-893e4ec69107" containerID="a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618" exitCode=0
Sep 15 12:46:14.241848 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:14.241715    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerDied","Data":"a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618"}
Sep 15 12:46:14.339454 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:14.339400    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw" podStartSLOduration=2.563652947 podStartE2EDuration="5.339379145s" totalImagesPullingTime="2.775726198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:14.280939963 +0000 UTC m=+92.073641693" watchObservedRunningTime="2026-09-15 12:46:14.339379145 +0000 UTC m=+92.132080875"
Sep 15 12:46:17.651295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:17.651263    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-788ffb67c9-gn56b"]
Sep 15 12:46:18.241135 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:18.241110    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-cccf48976-k6cbj"
Sep 15 12:46:18.266513 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:18.266483    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerStarted","Data":"c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518"}
Sep 15 12:46:18.266694 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:18.266551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerStarted","Data":"e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7"}
Sep 15 12:46:18.266694 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:18.266580    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerStarted","Data":"30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c"}
Sep 15 12:46:18.266694 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:18.266604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerStarted","Data":"89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482"}
Sep 15 12:46:18.266694 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:18.266635    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerStarted","Data":"07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4"}
Sep 15 12:46:18.266694 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:18.266663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerStarted","Data":"76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b"}
Sep 15 12:46:18.345788 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:18.345731    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.949031114 podStartE2EDuration="7.345713907s" totalImagesPullingTime="4.376112625s" totalInitContainerRuntime="20.570168ms" isStatefulPod=false podCreationTimestamp="2026-09-15 12:46:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:46:18.345320921 +0000 UTC m=+96.138022652" watchObservedRunningTime="2026-09-15 12:46:18.345713907 +0000 UTC m=+96.138415638"
Sep 15 12:46:21.105923 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:21.105892    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sm5v4"
Sep 15 12:46:22.116679 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:22.116645    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:46:30.301772 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:30.301731    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:30.302201 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:30.301801    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:33.334572 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:33.334486    2562 generic.go:356] "Generic (PLEG): container finished" podID="255a09b1-7274-435a-8ad0-59ba327862a9" containerID="d6ba220bd494c375113b0c0784034b768a4f4e2cad5089e6906c00a5e057bb77" exitCode=0
Sep 15 12:46:33.334572 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:33.334563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm" event={"ID":"255a09b1-7274-435a-8ad0-59ba327862a9","Type":"ContainerDied","Data":"d6ba220bd494c375113b0c0784034b768a4f4e2cad5089e6906c00a5e057bb77"}
Sep 15 12:46:33.334956 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:33.334882    2562 scope.go:117] "RemoveContainer" containerID="d6ba220bd494c375113b0c0784034b768a4f4e2cad5089e6906c00a5e057bb77"
Sep 15 12:46:34.340954 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:34.340919    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-bz6pm" event={"ID":"255a09b1-7274-435a-8ad0-59ba327862a9","Type":"ContainerStarted","Data":"1700ea730809abf553a4f2627e9dcd448eb7d3a7a2d0604518ae5ee840841a46"}
Sep 15 12:46:42.673061 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:42.672988    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b" podUID="c1542bbc-a7c6-4873-8672-6607a20f7632" containerName="registry" containerID="cri-o://4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744" gracePeriod=30
Sep 15 12:46:42.914979 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:42.914954    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:46:43.044871 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.044793    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frfpq\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-kube-api-access-frfpq\") pod \"c1542bbc-a7c6-4873-8672-6607a20f7632\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") "
Sep 15 12:46:43.044871 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.044843    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1542bbc-a7c6-4873-8672-6607a20f7632-ca-trust-extracted\") pod \"c1542bbc-a7c6-4873-8672-6607a20f7632\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") "
Sep 15 12:46:43.045126 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.044874    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-trusted-ca\") pod \"c1542bbc-a7c6-4873-8672-6607a20f7632\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") "
Sep 15 12:46:43.045126 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.044942    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") pod \"c1542bbc-a7c6-4873-8672-6607a20f7632\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") "
Sep 15 12:46:43.045126 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.044986    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-certificates\") pod \"c1542bbc-a7c6-4873-8672-6607a20f7632\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") "
Sep 15 12:46:43.045126 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.045063    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-bound-sa-token\") pod \"c1542bbc-a7c6-4873-8672-6607a20f7632\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") "
Sep 15 12:46:43.045126 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.045094    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-installation-pull-secrets\") pod \"c1542bbc-a7c6-4873-8672-6607a20f7632\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") "
Sep 15 12:46:43.045417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.045129    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-image-registry-private-configuration\") pod \"c1542bbc-a7c6-4873-8672-6607a20f7632\" (UID: \"c1542bbc-a7c6-4873-8672-6607a20f7632\") "
Sep 15 12:46:43.045519 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.045476    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c1542bbc-a7c6-4873-8672-6607a20f7632" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:46:43.045735 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.045690    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c1542bbc-a7c6-4873-8672-6607a20f7632" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:46:43.047743 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.047713    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c1542bbc-a7c6-4873-8672-6607a20f7632" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:46:43.047833 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.047774    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-kube-api-access-frfpq" (OuterVolumeSpecName: "kube-api-access-frfpq") pod "c1542bbc-a7c6-4873-8672-6607a20f7632" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632"). InnerVolumeSpecName "kube-api-access-frfpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:43.047896 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.047869    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c1542bbc-a7c6-4873-8672-6607a20f7632" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:43.047958 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.047934    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c1542bbc-a7c6-4873-8672-6607a20f7632" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:46:43.048012 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.047958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c1542bbc-a7c6-4873-8672-6607a20f7632" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:46:43.053377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.053355    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1542bbc-a7c6-4873-8672-6607a20f7632-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c1542bbc-a7c6-4873-8672-6607a20f7632" (UID: "c1542bbc-a7c6-4873-8672-6607a20f7632"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:46:43.146325 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.146298    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-bound-sa-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:46:43.146325 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.146322    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-installation-pull-secrets\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:46:43.146477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.146333    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1542bbc-a7c6-4873-8672-6607a20f7632-image-registry-private-configuration\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:46:43.146477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.146343    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frfpq\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-kube-api-access-frfpq\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:46:43.146477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.146351    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1542bbc-a7c6-4873-8672-6607a20f7632-ca-trust-extracted\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:46:43.146477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.146361    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-trusted-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:46:43.146477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.146369    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:46:43.146477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.146376    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1542bbc-a7c6-4873-8672-6607a20f7632-registry-certificates\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:46:43.377965 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.377883    2562 generic.go:356] "Generic (PLEG): container finished" podID="c1542bbc-a7c6-4873-8672-6607a20f7632" containerID="4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744" exitCode=0
Sep 15 12:46:43.377965 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.377932    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b"
Sep 15 12:46:43.378169 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.377968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b" event={"ID":"c1542bbc-a7c6-4873-8672-6607a20f7632","Type":"ContainerDied","Data":"4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744"}
Sep 15 12:46:43.378169 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.378020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788ffb67c9-gn56b" event={"ID":"c1542bbc-a7c6-4873-8672-6607a20f7632","Type":"ContainerDied","Data":"26dd957bc87e4fd56aeac958d4898e648fa01cf743ed64994460c32c6ea8389d"}
Sep 15 12:46:43.378169 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.378061    2562 scope.go:117] "RemoveContainer" containerID="4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744"
Sep 15 12:46:43.387517 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.387499    2562 scope.go:117] "RemoveContainer" containerID="4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744"
Sep 15 12:46:43.387787 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:46:43.387760    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744\": container with ID starting with 4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744 not found: ID does not exist" containerID="4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744"
Sep 15 12:46:43.387836 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.387795    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744"} err="failed to get container status \"4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744\": rpc error: code = NotFound desc = could not find container \"4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744\": container with ID starting with 4a42afcbdf2a9f408a8901cd5fe31c2d08d4b86b305715ae2690231dcfb73744 not found: ID does not exist"
Sep 15 12:46:43.418575 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.418542    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-788ffb67c9-gn56b"]
Sep 15 12:46:43.428601 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:43.428577    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-788ffb67c9-gn56b"]
Sep 15 12:46:44.755780 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:44.755741    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1542bbc-a7c6-4873-8672-6607a20f7632" path="/var/lib/kubelet/pods/c1542bbc-a7c6-4873-8672-6607a20f7632/volumes"
Sep 15 12:46:50.305972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:50.305946    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:50.309664 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:50.309639    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-784857c55d-j5dnw"
Sep 15 12:46:53.420862 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:53.420822    2562 generic.go:356] "Generic (PLEG): container finished" podID="d8d2cf54-fdd9-4336-9988-12725adb2d5a" containerID="061b0ced445102de68932ede63c05cd1088adcb6c4a445acd67929bfdd1589db" exitCode=0
Sep 15 12:46:53.421360 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:53.420899    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq" event={"ID":"d8d2cf54-fdd9-4336-9988-12725adb2d5a","Type":"ContainerDied","Data":"061b0ced445102de68932ede63c05cd1088adcb6c4a445acd67929bfdd1589db"}
Sep 15 12:46:53.421360 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:53.421336    2562 scope.go:117] "RemoveContainer" containerID="061b0ced445102de68932ede63c05cd1088adcb6c4a445acd67929bfdd1589db"
Sep 15 12:46:54.426605 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:46:54.426557    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-kdnnq" event={"ID":"d8d2cf54-fdd9-4336-9988-12725adb2d5a","Type":"ContainerStarted","Data":"aad3210467440be9f6cc0e01f6430d54aec95aa5bbcd72914da8def23ac585b9"}
Sep 15 12:47:03.466799 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:03.466764    2562 generic.go:356] "Generic (PLEG): container finished" podID="518a335e-be83-4292-82e7-fd571fc39ad1" containerID="24a4fd0cd91a994ca5e43f06ee06cb8d5c1f75895c4696b4e30cad2b2f2fb921" exitCode=0
Sep 15 12:47:03.467237 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:03.466837    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qh8lp" event={"ID":"518a335e-be83-4292-82e7-fd571fc39ad1","Type":"ContainerDied","Data":"24a4fd0cd91a994ca5e43f06ee06cb8d5c1f75895c4696b4e30cad2b2f2fb921"}
Sep 15 12:47:03.467237 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:03.467200    2562 scope.go:117] "RemoveContainer" containerID="24a4fd0cd91a994ca5e43f06ee06cb8d5c1f75895c4696b4e30cad2b2f2fb921"
Sep 15 12:47:04.472342 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:04.472311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qh8lp" event={"ID":"518a335e-be83-4292-82e7-fd571fc39ad1","Type":"ContainerStarted","Data":"151fdebaba4fd471e25506da02067cc10aa6683a9a4026aca692eb1604bb1bbd"}
Sep 15 12:47:12.116477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:12.116446    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:12.131920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:12.131898    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:12.521554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:12.521525    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:30.307642 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.307604    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:47:30.308182 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.307932    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="prometheus" containerID="cri-o://76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b" gracePeriod=600
Sep 15 12:47:30.308182 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.307947    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy" containerID="cri-o://e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7" gracePeriod=600
Sep 15 12:47:30.308182 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.308104    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy-web" containerID="cri-o://30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c" gracePeriod=600
Sep 15 12:47:30.308182 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.308135    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518" gracePeriod=600
Sep 15 12:47:30.308182 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.308178    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="thanos-sidecar" containerID="cri-o://89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482" gracePeriod=600
Sep 15 12:47:30.308478 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.308213    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="config-reloader" containerID="cri-o://07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4" gracePeriod=600
Sep 15 12:47:30.590614 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590527    2562 generic.go:356] "Generic (PLEG): container finished" podID="390c18d0-e350-4936-b3fd-893e4ec69107" containerID="c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518" exitCode=0
Sep 15 12:47:30.590614 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590552    2562 generic.go:356] "Generic (PLEG): container finished" podID="390c18d0-e350-4936-b3fd-893e4ec69107" containerID="e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7" exitCode=0
Sep 15 12:47:30.590614 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590562    2562 generic.go:356] "Generic (PLEG): container finished" podID="390c18d0-e350-4936-b3fd-893e4ec69107" containerID="89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482" exitCode=0
Sep 15 12:47:30.590614 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590570    2562 generic.go:356] "Generic (PLEG): container finished" podID="390c18d0-e350-4936-b3fd-893e4ec69107" containerID="07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4" exitCode=0
Sep 15 12:47:30.590614 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590576    2562 generic.go:356] "Generic (PLEG): container finished" podID="390c18d0-e350-4936-b3fd-893e4ec69107" containerID="76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b" exitCode=0
Sep 15 12:47:30.590614 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590596    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerDied","Data":"c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518"}
Sep 15 12:47:30.590902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerDied","Data":"e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7"}
Sep 15 12:47:30.590902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerDied","Data":"89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482"}
Sep 15 12:47:30.590902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590714    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerDied","Data":"07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4"}
Sep 15 12:47:30.590902 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:30.590738    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerDied","Data":"76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b"}
Sep 15 12:47:31.547662 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.547637    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:31.599575 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.599545    2562 generic.go:356] "Generic (PLEG): container finished" podID="390c18d0-e350-4936-b3fd-893e4ec69107" containerID="30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c" exitCode=0
Sep 15 12:47:31.599743 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.599595    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerDied","Data":"30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c"}
Sep 15 12:47:31.599743 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.599633    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:31.599743 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.599657    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"390c18d0-e350-4936-b3fd-893e4ec69107","Type":"ContainerDied","Data":"0c08be2bca331998956bbe990b4b2d66fdb174f90f56be8c465e90d1d4baea7b"}
Sep 15 12:47:31.599743 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.599691    2562 scope.go:117] "RemoveContainer" containerID="c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518"
Sep 15 12:47:31.610066 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.610046    2562 scope.go:117] "RemoveContainer" containerID="e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7"
Sep 15 12:47:31.618567 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.618546    2562 scope.go:117] "RemoveContainer" containerID="30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c"
Sep 15 12:47:31.625946 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.625922    2562 scope.go:117] "RemoveContainer" containerID="89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482"
Sep 15 12:47:31.633977 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.633959    2562 scope.go:117] "RemoveContainer" containerID="07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4"
Sep 15 12:47:31.640640 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.640622    2562 scope.go:117] "RemoveContainer" containerID="76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b"
Sep 15 12:47:31.647611 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.647594    2562 scope.go:117] "RemoveContainer" containerID="a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618"
Sep 15 12:47:31.651796 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.651777    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bw7w\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-kube-api-access-4bw7w\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.651904 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.651826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-kube-rbac-proxy\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.651904 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.651848    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-metrics-client-certs\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.651904 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.651869    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-1\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.651904 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.651890    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-metrics-client-ca\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.651920    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-serving-certs-ca-bundle\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.651956    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-tls\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.651987    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-0\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-thanos-prometheus-http-client-file\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652059    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652105    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-config-out\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652131    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652163    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-web-config\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652209    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-tls-assets\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652277    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-2\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652300    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-config\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652324    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-db\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652392    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-kubelet-serving-ca-bundle\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652427    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-grpc-tls\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.652815 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.652453    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-trusted-ca-bundle\") pod \"390c18d0-e350-4936-b3fd-893e4ec69107\" (UID: \"390c18d0-e350-4936-b3fd-893e4ec69107\") "
Sep 15 12:47:31.653204 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.653047    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:47:31.653204 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.653087    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:47:31.653593 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.653469    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:47:31.653593 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.653485    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:47:31.655647 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.653884    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:47:31.656198 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.655851    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:47:31.656198 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.655673    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:47:31.656198 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.655972    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.656198 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.655979    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-kube-api-access-4bw7w" (OuterVolumeSpecName: "kube-api-access-4bw7w") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "kube-api-access-4bw7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:47:31.656198 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.656154    2562 scope.go:117] "RemoveContainer" containerID="c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518"
Sep 15 12:47:31.656488 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:47:31.656459    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518\": container with ID starting with c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518 not found: ID does not exist" containerID="c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518"
Sep 15 12:47:31.656558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.656502    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518"} err="failed to get container status \"c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518\": rpc error: code = NotFound desc = could not find container \"c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518\": container with ID starting with c89f5d4fbe87a761a7701e425a9066acd2092e61415c01289a0571b2d7d2e518 not found: ID does not exist"
Sep 15 12:47:31.656558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.656530    2562 scope.go:117] "RemoveContainer" containerID="e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7"
Sep 15 12:47:31.656558 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.656469    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:47:31.656756 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.656682    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.656872 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:47:31.656839    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7\": container with ID starting with e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7 not found: ID does not exist" containerID="e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7"
Sep 15 12:47:31.656977 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.656879    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7"} err="failed to get container status \"e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7\": rpc error: code = NotFound desc = could not find container \"e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7\": container with ID starting with e22652413b1d366309d5739324ca89ec96ad076b71991086fc403e1ae4cf8cc7 not found: ID does not exist"
Sep 15 12:47:31.656977 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.656901    2562 scope.go:117] "RemoveContainer" containerID="30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c"
Sep 15 12:47:31.657176 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.657143    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-config" (OuterVolumeSpecName: "config") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.657375 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.657349    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.657569 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.657541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.657903 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:47:31.657754    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c\": container with ID starting with 30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c not found: ID does not exist" containerID="30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c"
Sep 15 12:47:31.657903 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.657795    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c"} err="failed to get container status \"30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c\": rpc error: code = NotFound desc = could not find container \"30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c\": container with ID starting with 30147ae1642f8a2f2f78b3ec34302ed1e86bb5d91082af028a9a2036e12bb15c not found: ID does not exist"
Sep 15 12:47:31.657903 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.657819    2562 scope.go:117] "RemoveContainer" containerID="89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482"
Sep 15 12:47:31.658354 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:47:31.658315    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482\": container with ID starting with 89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482 not found: ID does not exist" containerID="89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482"
Sep 15 12:47:31.658421 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.658364    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482"} err="failed to get container status \"89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482\": rpc error: code = NotFound desc = could not find container \"89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482\": container with ID starting with 89629729c23f9c1d2c51bb264d9852b5a826bb3afd6f1bf2b1d1dd19556cc482 not found: ID does not exist"
Sep 15 12:47:31.658421 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.658389    2562 scope.go:117] "RemoveContainer" containerID="07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4"
Sep 15 12:47:31.658728 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:47:31.658699    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4\": container with ID starting with 07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4 not found: ID does not exist" containerID="07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4"
Sep 15 12:47:31.658797 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.658735    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4"} err="failed to get container status \"07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4\": rpc error: code = NotFound desc = could not find container \"07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4\": container with ID starting with 07f4e01fa10c521dc032eab0091abff5a137fd2b8b320ec0b500a995d28fabd4 not found: ID does not exist"
Sep 15 12:47:31.658797 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.658757    2562 scope.go:117] "RemoveContainer" containerID="76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b"
Sep 15 12:47:31.658797 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.658763    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.659122 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:47:31.659095    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b\": container with ID starting with 76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b not found: ID does not exist" containerID="76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b"
Sep 15 12:47:31.659217 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.659127    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b"} err="failed to get container status \"76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b\": rpc error: code = NotFound desc = could not find container \"76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b\": container with ID starting with 76092a6e0495b11ef860da8b3e03a12aeb107cd4a969803007986f2bc273310b not found: ID does not exist"
Sep 15 12:47:31.659217 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.659146    2562 scope.go:117] "RemoveContainer" containerID="a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618"
Sep 15 12:47:31.659439 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:47:31.659380    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618\": container with ID starting with a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618 not found: ID does not exist" containerID="a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618"
Sep 15 12:47:31.659439 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.659389    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-config-out" (OuterVolumeSpecName: "config-out") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:47:31.659439 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.659400    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.659439 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.659409    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618"} err="failed to get container status \"a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618\": rpc error: code = NotFound desc = could not find container \"a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618\": container with ID starting with a64f1c76fb89838e8bfb2ca207256c020f421e3d134570f706af340de7c8e618 not found: ID does not exist"
Sep 15 12:47:31.659439 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.659392    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:47:31.659640 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.659573    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.670619 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.670594    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-web-config" (OuterVolumeSpecName: "web-config") pod "390c18d0-e350-4936-b3fd-893e4ec69107" (UID: "390c18d0-e350-4936-b3fd-893e4ec69107"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:47:31.753170 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753147    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-metrics-client-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753170 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753168    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753178    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-metrics-client-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753189    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753197    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753205    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753213    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753222    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753231    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-config-out\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753240    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753248    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-web-config\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753256    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-tls-assets\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753265    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753273    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-config\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753282    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-k8s-db\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753285 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753290    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753692 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753299    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-grpc-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753692 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753307    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390c18d0-e350-4936-b3fd-893e4ec69107-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753692 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753315    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bw7w\" (UniqueName: \"kubernetes.io/projected/390c18d0-e350-4936-b3fd-893e4ec69107-kube-api-access-4bw7w\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.753692 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.753323    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/390c18d0-e350-4936-b3fd-893e4ec69107-secret-kube-rbac-proxy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:47:31.931860 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.931831    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:47:31.935750 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.935726    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:47:31.970780 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.970754    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:47:31.971069 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971055    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="config-reloader"
Sep 15 12:47:31.971119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971070    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="config-reloader"
Sep 15 12:47:31.971119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971084    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1542bbc-a7c6-4873-8672-6607a20f7632" containerName="registry"
Sep 15 12:47:31.971119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971089    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1542bbc-a7c6-4873-8672-6607a20f7632" containerName="registry"
Sep 15 12:47:31.971119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971099    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="init-config-reloader"
Sep 15 12:47:31.971119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971104    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="init-config-reloader"
Sep 15 12:47:31.971119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971111    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="prometheus"
Sep 15 12:47:31.971119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971116    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="prometheus"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971124    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy-web"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971129    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy-web"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971143    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971148    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971154    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="thanos-sidecar"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971160    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="thanos-sidecar"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971169    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy-thanos"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971175    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy-thanos"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971229    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1542bbc-a7c6-4873-8672-6607a20f7632" containerName="registry"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971237    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971244    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="config-reloader"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971250    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy-web"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971258    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="kube-rbac-proxy-thanos"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971265    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="prometheus"
Sep 15 12:47:31.971334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.971270    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" containerName="thanos-sidecar"
Sep 15 12:47:31.976542 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.976521    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:31.981679 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.981659    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 12:47:31.982112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.982092    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 12:47:31.982217 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.982198    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 12:47:31.982279 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.982271    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 12:47:31.985851 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.985835    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 12:47:31.985851 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.985843    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 12:47:31.985977 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.985855    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 12:47:31.986343 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.986324    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 12:47:31.986439 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.986335    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 12:47:31.986544 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.986531    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 12:47:31.986594 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.986584    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 12:47:31.986650 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.986617    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 12:47:31.987106 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.987090    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f7mid7225h89a\""
Sep 15 12:47:31.987186 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.987131    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l8z4s\""
Sep 15 12:47:31.988545 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.988526    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 12:47:31.992239 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:31.992224    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 12:47:32.008317 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.008240    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:47:32.055722 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055695    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.055890 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055726    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.055890 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.055890 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055769    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.055890 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055795    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.055890 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055813    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.055890 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.055890 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055846    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.055890 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055861    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-web-config\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.055995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056047    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj9tl\" (UniqueName: \"kubernetes.io/projected/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-kube-api-access-gj9tl\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056144    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-config-out\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056166    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-config\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056197    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056213    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056538 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.056538 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.056265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156660 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-config-out\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156660 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-config\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156660 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156674    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156761    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156840    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156866    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.156949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156949    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.157344 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.157344 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.156986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.157344 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.157011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.158123 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.158092    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.158344 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.158292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.158414 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.158354    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.158952 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.158925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.159068 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.159052    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-web-config\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.159131 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.159104    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.159196 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.159137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj9tl\" (UniqueName: \"kubernetes.io/projected/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-kube-api-access-gj9tl\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.159196 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.159174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.160841 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.160329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-config-out\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.160841 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.160509    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.161116 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.161091    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.162508 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.161824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.162508 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.162106    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.162689 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.162533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-config\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.163139 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.163088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-web-config\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.163535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.163439    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.164061 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.164000    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.167331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.164282    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.167331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.164539    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.167331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.164623    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.167331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.164884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.167331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.165229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.167331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.165789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.172294 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.172271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj9tl\" (UniqueName: \"kubernetes.io/projected/8785bb8a-ab86-40d5-bfd0-f330dcf88b30-kube-api-access-gj9tl\") pod \"prometheus-k8s-0\" (UID: \"8785bb8a-ab86-40d5-bfd0-f330dcf88b30\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.285731 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.285703    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:47:32.575437 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.575412    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 12:47:32.577736 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:47:32.577706    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8785bb8a_ab86_40d5_bfd0_f330dcf88b30.slice/crio-40898dc8a55fd502d209cbae28177f14a993d69393749c2a7c828766955fd9c0 WatchSource:0}: Error finding container 40898dc8a55fd502d209cbae28177f14a993d69393749c2a7c828766955fd9c0: Status 404 returned error can't find the container with id 40898dc8a55fd502d209cbae28177f14a993d69393749c2a7c828766955fd9c0
Sep 15 12:47:32.605821 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.605786    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8785bb8a-ab86-40d5-bfd0-f330dcf88b30","Type":"ContainerStarted","Data":"40898dc8a55fd502d209cbae28177f14a993d69393749c2a7c828766955fd9c0"}
Sep 15 12:47:32.753921 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:32.753893    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="390c18d0-e350-4936-b3fd-893e4ec69107" path="/var/lib/kubelet/pods/390c18d0-e350-4936-b3fd-893e4ec69107/volumes"
Sep 15 12:47:33.612729 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:33.612692    2562 generic.go:356] "Generic (PLEG): container finished" podID="8785bb8a-ab86-40d5-bfd0-f330dcf88b30" containerID="0f675b4f4e9e0582e71defb935a70f8383e2bd9196b41580b0c60daec6b50206" exitCode=0
Sep 15 12:47:33.613090 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:33.612780    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8785bb8a-ab86-40d5-bfd0-f330dcf88b30","Type":"ContainerDied","Data":"0f675b4f4e9e0582e71defb935a70f8383e2bd9196b41580b0c60daec6b50206"}
Sep 15 12:47:34.623106 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:34.623072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8785bb8a-ab86-40d5-bfd0-f330dcf88b30","Type":"ContainerStarted","Data":"f090bf8d7c679d79315760835dd97191f1d3534a776825f0ab418ed9b25939ee"}
Sep 15 12:47:34.623472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:34.623136    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8785bb8a-ab86-40d5-bfd0-f330dcf88b30","Type":"ContainerStarted","Data":"0fdeb5ce6e3ece3685bd1c151d72c7ad156971f97c94da6f506aad284df30131"}
Sep 15 12:47:34.623472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:34.623161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8785bb8a-ab86-40d5-bfd0-f330dcf88b30","Type":"ContainerStarted","Data":"9b4c1fed51b47574268b45947659839800a54267d38d4a528d7473078423018e"}
Sep 15 12:47:34.623472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:34.623191    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8785bb8a-ab86-40d5-bfd0-f330dcf88b30","Type":"ContainerStarted","Data":"fa2b55fcdceb0a07acd018475dc5cd4cfd7b25a5b8796ca2f215ed739097b90f"}
Sep 15 12:47:34.623472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:34.623215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8785bb8a-ab86-40d5-bfd0-f330dcf88b30","Type":"ContainerStarted","Data":"f9d133954c7efc3e6d0527e515299510675dd55d77824547afccc1c7a06457fd"}
Sep 15 12:47:34.623472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:34.623237    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8785bb8a-ab86-40d5-bfd0-f330dcf88b30","Type":"ContainerStarted","Data":"25c566ff260cb42647164b3661f0e141bee4e2b72b0cf3254a56b39c59d9c8e6"}
Sep 15 12:47:34.661689 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:34.661616    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.6452964740000002 podStartE2EDuration="3.661595216s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.298742ms" isStatefulPod=false podCreationTimestamp="2026-09-15 12:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:47:34.659415234 +0000 UTC m=+172.452116967" watchObservedRunningTime="2026-09-15 12:47:34.661595216 +0000 UTC m=+172.454296950"
Sep 15 12:47:37.286111 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:47:37.286017    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:48:32.286451 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:48:32.286406    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:48:32.301925 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:48:32.301900    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:48:32.868766 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:48:32.868741    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 12:49:42.631293 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:49:42.631262    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:49:42.634492 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:49:42.634465    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:49:42.635460 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:49:42.635436    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:49:42.639109 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:49:42.639088    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:49:42.642700 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:49:42.642683    2562 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 12:51:26.631610 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.631573    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"]
Sep 15 12:51:26.635128 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.635093    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.644438 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.644418    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 12:51:26.646229 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.646213    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 12:51:26.646435 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.646422    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 12:51:26.646574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.646558    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g67c7\""
Sep 15 12:51:26.646669 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.646655    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 12:51:26.666922 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.666898    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"]
Sep 15 12:51:26.783798 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.783766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/653dfa3e-9221-4267-b320-e23bfe775222-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.783965 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.783820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkhmt\" (UniqueName: \"kubernetes.io/projected/653dfa3e-9221-4267-b320-e23bfe775222-kube-api-access-qkhmt\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.783965 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.783842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/653dfa3e-9221-4267-b320-e23bfe775222-webhook-cert\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.784091 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.783953    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/653dfa3e-9221-4267-b320-e23bfe775222-manifests\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.784091 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.784019    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/653dfa3e-9221-4267-b320-e23bfe775222-tmp\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.884870 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.884792    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/653dfa3e-9221-4267-b320-e23bfe775222-manifests\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.884870 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.884824    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/653dfa3e-9221-4267-b320-e23bfe775222-tmp\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.884870 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.884852    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/653dfa3e-9221-4267-b320-e23bfe775222-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.885170 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.884895    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkhmt\" (UniqueName: \"kubernetes.io/projected/653dfa3e-9221-4267-b320-e23bfe775222-kube-api-access-qkhmt\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.885170 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.884919    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/653dfa3e-9221-4267-b320-e23bfe775222-webhook-cert\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.885278 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.885258    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/653dfa3e-9221-4267-b320-e23bfe775222-manifests\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.887200 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.887167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/653dfa3e-9221-4267-b320-e23bfe775222-tmp\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.887398 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.887378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/653dfa3e-9221-4267-b320-e23bfe775222-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.887450 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.887431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/653dfa3e-9221-4267-b320-e23bfe775222-webhook-cert\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.894060 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.894017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkhmt\" (UniqueName: \"kubernetes.io/projected/653dfa3e-9221-4267-b320-e23bfe775222-kube-api-access-qkhmt\") pod \"opendatahub-operator-controller-manager-79946fddd6-l9vd4\" (UID: \"653dfa3e-9221-4267-b320-e23bfe775222\") " pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:26.945389 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:26.945368    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:27.222823 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:27.222793    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"]
Sep 15 12:51:27.225565 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:51:27.225541    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653dfa3e_9221_4267_b320_e23bfe775222.slice/crio-9d23beda0a0554e56a42f128ed03b4bb99e081757d78b123b992c4d1ae715711 WatchSource:0}: Error finding container 9d23beda0a0554e56a42f128ed03b4bb99e081757d78b123b992c4d1ae715711: Status 404 returned error can't find the container with id 9d23beda0a0554e56a42f128ed03b4bb99e081757d78b123b992c4d1ae715711
Sep 15 12:51:27.227479 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:27.227459    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:51:27.552930 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:27.552844    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4" event={"ID":"653dfa3e-9221-4267-b320-e23bfe775222","Type":"ContainerStarted","Data":"9d23beda0a0554e56a42f128ed03b4bb99e081757d78b123b992c4d1ae715711"}
Sep 15 12:51:30.568188 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:30.568102    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4" event={"ID":"653dfa3e-9221-4267-b320-e23bfe775222","Type":"ContainerStarted","Data":"fccf28c2ba32a3e63aff4d068fc23cecd4c557823a2eb1ea4b251c5ce7703173"}
Sep 15 12:51:31.574228 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:31.574194    2562 generic.go:356] "Generic (PLEG): container finished" podID="653dfa3e-9221-4267-b320-e23bfe775222" containerID="fccf28c2ba32a3e63aff4d068fc23cecd4c557823a2eb1ea4b251c5ce7703173" exitCode=0
Sep 15 12:51:31.574619 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:31.574238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4" event={"ID":"653dfa3e-9221-4267-b320-e23bfe775222","Type":"ContainerDied","Data":"fccf28c2ba32a3e63aff4d068fc23cecd4c557823a2eb1ea4b251c5ce7703173"}
Sep 15 12:51:32.581156 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:32.581062    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4" event={"ID":"653dfa3e-9221-4267-b320-e23bfe775222","Type":"ContainerStarted","Data":"53f33373d7420c7364e73922539ba8b9aafc7bffa152832e0f29f61c2f774f59"}
Sep 15 12:51:32.581535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:32.581156    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:32.629970 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:32.629914    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4" podStartSLOduration=1.947515267 podStartE2EDuration="6.629893929s" totalImagesPullingTime="3.632702673s" totalInitContainerRuntime="1.049675989s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:51:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:51:32.628892867 +0000 UTC m=+410.421594599" watchObservedRunningTime="2026-09-15 12:51:32.629893929 +0000 UTC m=+410.422595661"
Sep 15 12:51:43.587472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:43.587439    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79946fddd6-l9vd4"
Sep 15 12:51:53.971495 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:53.971459    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"]
Sep 15 12:51:53.978709 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:53.978683    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:53.984335 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:53.984315    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-86wcw\""
Sep 15 12:51:53.985232 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:53.985116    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 12:51:53.994874 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:53.994852    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"]
Sep 15 12:51:54.135258 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.135258 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.135509 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a531882-df52-4750-a943-2153d5ae86f9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.135509 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.135509 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135473    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmg6z\" (UniqueName: \"kubernetes.io/projected/7a531882-df52-4750-a943-2153d5ae86f9-kube-api-access-xmg6z\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.135658 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.135658 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a531882-df52-4750-a943-2153d5ae86f9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.135658 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.135658 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.135614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a531882-df52-4750-a943-2153d5ae86f9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.236813 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.236813 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.236813 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a531882-df52-4750-a943-2153d5ae86f9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237167 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237167 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmg6z\" (UniqueName: \"kubernetes.io/projected/7a531882-df52-4750-a943-2153d5ae86f9-kube-api-access-xmg6z\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237167 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237167 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a531882-df52-4750-a943-2153d5ae86f9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237167 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236940    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237167 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.236975    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a531882-df52-4750-a943-2153d5ae86f9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.237260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.237383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237805 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.237781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.237949 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.237924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a531882-df52-4750-a943-2153d5ae86f9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.238059 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.237978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.239731 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.239708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a531882-df52-4750-a943-2153d5ae86f9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.239967 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.239947    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a531882-df52-4750-a943-2153d5ae86f9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.251205 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.251178    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a531882-df52-4750-a943-2153d5ae86f9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.253557 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.253535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmg6z\" (UniqueName: \"kubernetes.io/projected/7a531882-df52-4750-a943-2153d5ae86f9-kube-api-access-xmg6z\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78\" (UID: \"7a531882-df52-4750-a943-2153d5ae86f9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.288824 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.288796    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:54.565744 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.565720    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"]
Sep 15 12:51:54.568055 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:51:54.568020    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a531882_df52_4750_a943_2153d5ae86f9.slice/crio-9c7c0972e49be582ecbab515208ca1acfba27b21a6a4bfdec15ea4165bffcb3a WatchSource:0}: Error finding container 9c7c0972e49be582ecbab515208ca1acfba27b21a6a4bfdec15ea4165bffcb3a: Status 404 returned error can't find the container with id 9c7c0972e49be582ecbab515208ca1acfba27b21a6a4bfdec15ea4165bffcb3a
Sep 15 12:51:54.683352 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:54.683307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78" event={"ID":"7a531882-df52-4750-a943-2153d5ae86f9","Type":"ContainerStarted","Data":"9c7c0972e49be582ecbab515208ca1acfba27b21a6a4bfdec15ea4165bffcb3a"}
Sep 15 12:51:57.026928 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.026886    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2pns4"]
Sep 15 12:51:57.034345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.034317    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:57.038391 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.038303    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7h4vc\""
Sep 15 12:51:57.043635 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.042797    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 15 12:51:57.050824 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.050795    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2pns4"]
Sep 15 12:51:57.166218 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.166172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-cert\") pod \"odh-model-controller-7998c67dc4-2pns4\" (UID: \"51f724a6-cbd4-47a3-86e7-ef5565c01ec3\") " pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:57.166396 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.166240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf6vs\" (UniqueName: \"kubernetes.io/projected/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-kube-api-access-qf6vs\") pod \"odh-model-controller-7998c67dc4-2pns4\" (UID: \"51f724a6-cbd4-47a3-86e7-ef5565c01ec3\") " pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:57.267403 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.267367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-cert\") pod \"odh-model-controller-7998c67dc4-2pns4\" (UID: \"51f724a6-cbd4-47a3-86e7-ef5565c01ec3\") " pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:57.267580 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.267422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf6vs\" (UniqueName: \"kubernetes.io/projected/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-kube-api-access-qf6vs\") pod \"odh-model-controller-7998c67dc4-2pns4\" (UID: \"51f724a6-cbd4-47a3-86e7-ef5565c01ec3\") " pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:57.267580 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:51:57.267524    2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 15 12:51:57.267700 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:51:57.267601    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-cert podName:51f724a6-cbd4-47a3-86e7-ef5565c01ec3 nodeName:}" failed. No retries permitted until 2026-09-15 12:51:57.767577952 +0000 UTC m=+435.560279721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-cert") pod "odh-model-controller-7998c67dc4-2pns4" (UID: "51f724a6-cbd4-47a3-86e7-ef5565c01ec3") : secret "odh-model-controller-webhook-cert" not found
Sep 15 12:51:57.277630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.277557    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf6vs\" (UniqueName: \"kubernetes.io/projected/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-kube-api-access-qf6vs\") pod \"odh-model-controller-7998c67dc4-2pns4\" (UID: \"51f724a6-cbd4-47a3-86e7-ef5565c01ec3\") " pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:57.510648 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.510607    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:57.510717 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.510684    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:57.510717 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.510712    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:51:57.701713 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.701675    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78" event={"ID":"7a531882-df52-4750-a943-2153d5ae86f9","Type":"ContainerStarted","Data":"c5dfbe8964fd987e8b0abb7547dbc9194c50aba980da826209f98c9a96ad14f5"}
Sep 15 12:51:57.737331 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.737267    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78" podStartSLOduration=1.79697376 podStartE2EDuration="4.737245532s" totalImagesPullingTime="2.940271772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:51:57.735732901 +0000 UTC m=+435.528434634" watchObservedRunningTime="2026-09-15 12:51:57.737245532 +0000 UTC m=+435.529947263"
Sep 15 12:51:57.771548 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.771507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-cert\") pod \"odh-model-controller-7998c67dc4-2pns4\" (UID: \"51f724a6-cbd4-47a3-86e7-ef5565c01ec3\") " pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:57.774173 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.774137    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51f724a6-cbd4-47a3-86e7-ef5565c01ec3-cert\") pod \"odh-model-controller-7998c67dc4-2pns4\" (UID: \"51f724a6-cbd4-47a3-86e7-ef5565c01ec3\") " pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:57.952664 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:57.952627    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:51:58.229951 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:58.229923    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2pns4"]
Sep 15 12:51:58.231929 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:51:58.231904    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51f724a6_cbd4_47a3_86e7_ef5565c01ec3.slice/crio-1c1de7f00488c390172825c4aa4e1e42ce9900d16da76a6463c54e29ba0e6fad WatchSource:0}: Error finding container 1c1de7f00488c390172825c4aa4e1e42ce9900d16da76a6463c54e29ba0e6fad: Status 404 returned error can't find the container with id 1c1de7f00488c390172825c4aa4e1e42ce9900d16da76a6463c54e29ba0e6fad
Sep 15 12:51:58.289358 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:58.289330    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:58.294118 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:58.294094    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:58.709068 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:58.709019    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2pns4" event={"ID":"51f724a6-cbd4-47a3-86e7-ef5565c01ec3","Type":"ContainerStarted","Data":"1c1de7f00488c390172825c4aa4e1e42ce9900d16da76a6463c54e29ba0e6fad"}
Sep 15 12:51:58.709263 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:58.709247    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:51:58.710281 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:51:58.710265    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78"
Sep 15 12:52:01.730221 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:01.730177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2pns4" event={"ID":"51f724a6-cbd4-47a3-86e7-ef5565c01ec3","Type":"ContainerStarted","Data":"67167e2e06cb4e605727400a005530798b2aa978e38740965946fc0af7084e39"}
Sep 15 12:52:01.730623 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:01.730267    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:52:01.751784 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:01.751721    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2pns4" podStartSLOduration=1.895438881 podStartE2EDuration="4.751698348s" totalImagesPullingTime="2.856259467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:52:01.749435391 +0000 UTC m=+439.542137137" watchObservedRunningTime="2026-09-15 12:52:01.751698348 +0000 UTC m=+439.544400082"
Sep 15 12:52:04.655520 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.655486    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"]
Sep 15 12:52:04.658956 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.658932    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.663351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.663332    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 15 12:52:04.663509 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.663349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 15 12:52:04.663509 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.663348    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 15 12:52:04.663680 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.663385    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gdw9j\""
Sep 15 12:52:04.663680 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.663383    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 15 12:52:04.663680 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.663405    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 15 12:52:04.669750 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.669727    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"]
Sep 15 12:52:04.844222 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.844194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9b5e605-355b-465a-a753-7d16282e513e-cert\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.844384 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.844228    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f9b5e605-355b-465a-a753-7d16282e513e-manager-config\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.844384 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.844252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9b5e605-355b-465a-a753-7d16282e513e-metrics-cert\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.844384 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.844270    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkk6q\" (UniqueName: \"kubernetes.io/projected/f9b5e605-355b-465a-a753-7d16282e513e-kube-api-access-bkk6q\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.945290 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.945248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9b5e605-355b-465a-a753-7d16282e513e-cert\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.945290 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.945297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f9b5e605-355b-465a-a753-7d16282e513e-manager-config\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.945515 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.945332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9b5e605-355b-465a-a753-7d16282e513e-metrics-cert\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.945515 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.945348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkk6q\" (UniqueName: \"kubernetes.io/projected/f9b5e605-355b-465a-a753-7d16282e513e-kube-api-access-bkk6q\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.946148 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.946115    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f9b5e605-355b-465a-a753-7d16282e513e-manager-config\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.947728 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.947708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9b5e605-355b-465a-a753-7d16282e513e-cert\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.947818 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.947754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9b5e605-355b-465a-a753-7d16282e513e-metrics-cert\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:04.973677 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:04.973653    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkk6q\" (UniqueName: \"kubernetes.io/projected/f9b5e605-355b-465a-a753-7d16282e513e-kube-api-access-bkk6q\") pod \"lws-controller-manager-c5fdc7d8d-6wsdf\" (UID: \"f9b5e605-355b-465a-a753-7d16282e513e\") " pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:05.269227 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:05.269150    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:05.566210 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:05.566185    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"]
Sep 15 12:52:05.569675 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:52:05.569637    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b5e605_355b_465a_a753_7d16282e513e.slice/crio-b66d0760c0da8afdc1a0578e30e604e00bf6d9056743590467f6de6db51739f5 WatchSource:0}: Error finding container b66d0760c0da8afdc1a0578e30e604e00bf6d9056743590467f6de6db51739f5: Status 404 returned error can't find the container with id b66d0760c0da8afdc1a0578e30e604e00bf6d9056743590467f6de6db51739f5
Sep 15 12:52:05.749660 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:05.749625    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf" event={"ID":"f9b5e605-355b-465a-a753-7d16282e513e","Type":"ContainerStarted","Data":"b66d0760c0da8afdc1a0578e30e604e00bf6d9056743590467f6de6db51739f5"}
Sep 15 12:52:08.203489 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.203461    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"]
Sep 15 12:52:08.212725 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.212698    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:08.216984 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.216962    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 15 12:52:08.219903 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.219883    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-m7qf9\""
Sep 15 12:52:08.241090 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.241068    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"]
Sep 15 12:52:08.242934 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.242912    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"]
Sep 15 12:52:08.246575 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.246557    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:08.262241 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.262212    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 15 12:52:08.262729 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.262711    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rcndl\""
Sep 15 12:52:08.274818 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.274788    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"]
Sep 15 12:52:08.276163 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.276135    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-flzms\" (UID: \"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:08.276300 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.276212    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsdn9\" (UniqueName: \"kubernetes.io/projected/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-kube-api-access-lsdn9\") pod \"llmisvc-controller-manager-777c6bd7d5-flzms\" (UID: \"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:08.276300 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.276244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jsfm\" (UniqueName: \"kubernetes.io/projected/91586217-bf10-4574-a510-9787484c3360-kube-api-access-4jsfm\") pod \"kserve-controller-manager-d6f4b65b8-gx4hh\" (UID: \"91586217-bf10-4574-a510-9787484c3360\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:08.276300 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.276284    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91586217-bf10-4574-a510-9787484c3360-cert\") pod \"kserve-controller-manager-d6f4b65b8-gx4hh\" (UID: \"91586217-bf10-4574-a510-9787484c3360\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:08.377631 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.377597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-flzms\" (UID: \"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:08.377787 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.377654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsdn9\" (UniqueName: \"kubernetes.io/projected/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-kube-api-access-lsdn9\") pod \"llmisvc-controller-manager-777c6bd7d5-flzms\" (UID: \"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:08.377787 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.377679    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jsfm\" (UniqueName: \"kubernetes.io/projected/91586217-bf10-4574-a510-9787484c3360-kube-api-access-4jsfm\") pod \"kserve-controller-manager-d6f4b65b8-gx4hh\" (UID: \"91586217-bf10-4574-a510-9787484c3360\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:08.377787 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:52:08.377684    2562 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 15 12:52:08.377787 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.377704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91586217-bf10-4574-a510-9787484c3360-cert\") pod \"kserve-controller-manager-d6f4b65b8-gx4hh\" (UID: \"91586217-bf10-4574-a510-9787484c3360\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:08.377787 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:52:08.377760    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-cert podName:98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e nodeName:}" failed. No retries permitted until 2026-09-15 12:52:08.877740385 +0000 UTC m=+446.670442115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-cert") pod "llmisvc-controller-manager-777c6bd7d5-flzms" (UID: "98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e") : secret "llmisvc-webhook-server-cert" not found
Sep 15 12:52:08.380044 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.380012    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91586217-bf10-4574-a510-9787484c3360-cert\") pod \"kserve-controller-manager-d6f4b65b8-gx4hh\" (UID: \"91586217-bf10-4574-a510-9787484c3360\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:08.394209 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.394188    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jsfm\" (UniqueName: \"kubernetes.io/projected/91586217-bf10-4574-a510-9787484c3360-kube-api-access-4jsfm\") pod \"kserve-controller-manager-d6f4b65b8-gx4hh\" (UID: \"91586217-bf10-4574-a510-9787484c3360\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:08.399064 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.399020    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsdn9\" (UniqueName: \"kubernetes.io/projected/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-kube-api-access-lsdn9\") pod \"llmisvc-controller-manager-777c6bd7d5-flzms\" (UID: \"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:08.522790 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.522710    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:08.767474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.767439    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf" event={"ID":"f9b5e605-355b-465a-a753-7d16282e513e","Type":"ContainerStarted","Data":"29ba2ac5c01ca220546ed9109ad3e653e65f7f23f0fdbf61e4cf928736418df8"}
Sep 15 12:52:08.767629 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.767585    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:08.829778 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.829679    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf" podStartSLOduration=2.486147182 podStartE2EDuration="4.829660371s" totalImagesPullingTime="2.343513189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:52:08.818711887 +0000 UTC m=+446.611413619" watchObservedRunningTime="2026-09-15 12:52:08.829660371 +0000 UTC m=+446.622362100"
Sep 15 12:52:08.831390 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.831357    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"]
Sep 15 12:52:08.836615 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:52:08.836565    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91586217_bf10_4574_a510_9787484c3360.slice/crio-ad2dfc8f160e7d356d8b2f1f872438843317273911923951b6ed787bb9d1d33b WatchSource:0}: Error finding container ad2dfc8f160e7d356d8b2f1f872438843317273911923951b6ed787bb9d1d33b: Status 404 returned error can't find the container with id ad2dfc8f160e7d356d8b2f1f872438843317273911923951b6ed787bb9d1d33b
Sep 15 12:52:08.881698 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.881646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-flzms\" (UID: \"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:08.884120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:08.884096    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-flzms\" (UID: \"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:09.176510 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:09.176477    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:09.452231 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:09.452152    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"]
Sep 15 12:52:09.455346 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:52:09.455318    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod98cfcc4e_4c7d_48dd_90f5_d6d763ea7d1e.slice/crio-65a9f9c3b6376b70e27f49840e958a0bfe5b09264a03be1edffd40bc67ab198f WatchSource:0}: Error finding container 65a9f9c3b6376b70e27f49840e958a0bfe5b09264a03be1edffd40bc67ab198f: Status 404 returned error can't find the container with id 65a9f9c3b6376b70e27f49840e958a0bfe5b09264a03be1edffd40bc67ab198f
Sep 15 12:52:09.774262 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:09.774176    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms" event={"ID":"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e","Type":"ContainerStarted","Data":"65a9f9c3b6376b70e27f49840e958a0bfe5b09264a03be1edffd40bc67ab198f"}
Sep 15 12:52:09.775572 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:09.775537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh" event={"ID":"91586217-bf10-4574-a510-9787484c3360","Type":"ContainerStarted","Data":"ad2dfc8f160e7d356d8b2f1f872438843317273911923951b6ed787bb9d1d33b"}
Sep 15 12:52:11.788439 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:11.788401    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh" event={"ID":"91586217-bf10-4574-a510-9787484c3360","Type":"ContainerStarted","Data":"90a76b37cc3a1836c609765039635853dce3f226d6cbfc3e19240481b07f9c8b"}
Sep 15 12:52:11.788840 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:11.788551    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:12.737400 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:12.737372    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2pns4"
Sep 15 12:52:12.757978 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:12.757889    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh" podStartSLOduration=2.014741615 podStartE2EDuration="4.757855686s" totalImagesPullingTime="2.743114071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:52:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:52:11.825760782 +0000 UTC m=+449.618462543" watchObservedRunningTime="2026-09-15 12:52:12.757855686 +0000 UTC m=+450.550557419"
Sep 15 12:52:13.801133 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:13.801092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms" event={"ID":"98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e","Type":"ContainerStarted","Data":"61fc49fc69432106614a58d2dc8d3eb664be17483937119bd4b54c97e4b989a9"}
Sep 15 12:52:13.801543 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:13.801216    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:52:13.831448 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:13.831403    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms" podStartSLOduration=1.62620495 podStartE2EDuration="5.831386315s" totalImagesPullingTime="4.205181365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:52:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:52:13.830099594 +0000 UTC m=+451.622801318" watchObservedRunningTime="2026-09-15 12:52:13.831386315 +0000 UTC m=+451.624088045"
Sep 15 12:52:19.777906 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:19.777868    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c5fdc7d8d-6wsdf"
Sep 15 12:52:42.800598 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:42.800570    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx4hh"
Sep 15 12:52:44.807918 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:52:44.807885    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-flzms"
Sep 15 12:53:20.213937 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.213899    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"]
Sep 15 12:53:20.216579 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.216558    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"
Sep 15 12:53:20.219465 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.219443    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 15 12:53:20.219609 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.219589    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8brxj\""
Sep 15 12:53:20.219860 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.219846    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 15 12:53:20.220529 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.220514    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 15 12:53:20.233587 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.233550    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"]
Sep 15 12:53:20.249752 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.249720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4d89\" (UniqueName: \"kubernetes.io/projected/48da0f6b-419d-4bef-841a-25ef4899e8bf-kube-api-access-h4d89\") pod \"dns-operator-controller-manager-7b84cd7c9c-4grfh\" (UID: \"48da0f6b-419d-4bef-841a-25ef4899e8bf\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"
Sep 15 12:53:20.350320 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.350281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4d89\" (UniqueName: \"kubernetes.io/projected/48da0f6b-419d-4bef-841a-25ef4899e8bf-kube-api-access-h4d89\") pod \"dns-operator-controller-manager-7b84cd7c9c-4grfh\" (UID: \"48da0f6b-419d-4bef-841a-25ef4899e8bf\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"
Sep 15 12:53:20.382113 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.382076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4d89\" (UniqueName: \"kubernetes.io/projected/48da0f6b-419d-4bef-841a-25ef4899e8bf-kube-api-access-h4d89\") pod \"dns-operator-controller-manager-7b84cd7c9c-4grfh\" (UID: \"48da0f6b-419d-4bef-841a-25ef4899e8bf\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"
Sep 15 12:53:20.527513 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.527422    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"
Sep 15 12:53:20.811384 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:20.811361    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"]
Sep 15 12:53:20.813866 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:53:20.813833    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48da0f6b_419d_4bef_841a_25ef4899e8bf.slice/crio-794571e78e7fd93b5971af7ca8af7994e87744805d0c7bf0ea2c032b691f5e0a WatchSource:0}: Error finding container 794571e78e7fd93b5971af7ca8af7994e87744805d0c7bf0ea2c032b691f5e0a: Status 404 returned error can't find the container with id 794571e78e7fd93b5971af7ca8af7994e87744805d0c7bf0ea2c032b691f5e0a
Sep 15 12:53:21.114111 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:21.113997    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh" event={"ID":"48da0f6b-419d-4bef-841a-25ef4899e8bf","Type":"ContainerStarted","Data":"794571e78e7fd93b5971af7ca8af7994e87744805d0c7bf0ea2c032b691f5e0a"}
Sep 15 12:53:24.133358 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:24.133317    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh" event={"ID":"48da0f6b-419d-4bef-841a-25ef4899e8bf","Type":"ContainerStarted","Data":"19ca24b8fa53b51f8b030be9515b8127347d4e9acfa647f278d8861e2e4373dd"}
Sep 15 12:53:24.133856 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:24.133451    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"
Sep 15 12:53:24.160931 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:24.160871    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh" podStartSLOduration=1.660422653 podStartE2EDuration="4.160853259s" totalImagesPullingTime="2.500430606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:53:24.160469394 +0000 UTC m=+521.953171126" watchObservedRunningTime="2026-09-15 12:53:24.160853259 +0000 UTC m=+521.953554990"
Sep 15 12:53:27.505826 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.505782    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"]
Sep 15 12:53:27.508520 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.508498    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:27.512023 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.512001    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-k7ddn\""
Sep 15 12:53:27.530314 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.530283    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"]
Sep 15 12:53:27.616317 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.616274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds67v\" (UniqueName: \"kubernetes.io/projected/c95a5ac1-aeb6-4caf-91b2-986f513c4097-kube-api-access-ds67v\") pod \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" (UID: \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:27.616511 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.616390    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c95a5ac1-aeb6-4caf-91b2-986f513c4097-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" (UID: \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:27.717501 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.717452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c95a5ac1-aeb6-4caf-91b2-986f513c4097-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" (UID: \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:27.717658 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.717536    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds67v\" (UniqueName: \"kubernetes.io/projected/c95a5ac1-aeb6-4caf-91b2-986f513c4097-kube-api-access-ds67v\") pod \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" (UID: \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:27.717947 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.717921    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c95a5ac1-aeb6-4caf-91b2-986f513c4097-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" (UID: \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:27.734417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.734378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds67v\" (UniqueName: \"kubernetes.io/projected/c95a5ac1-aeb6-4caf-91b2-986f513c4097-kube-api-access-ds67v\") pod \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" (UID: \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:27.818780 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:27.818692    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:28.100766 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:28.100695    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"]
Sep 15 12:53:28.104894 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:53:28.104861    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95a5ac1_aeb6_4caf_91b2_986f513c4097.slice/crio-1b627765bac466beda84fceed97eb0e29d75d5a4974c5ec23e0b3b15710b7e15 WatchSource:0}: Error finding container 1b627765bac466beda84fceed97eb0e29d75d5a4974c5ec23e0b3b15710b7e15: Status 404 returned error can't find the container with id 1b627765bac466beda84fceed97eb0e29d75d5a4974c5ec23e0b3b15710b7e15
Sep 15 12:53:28.158013 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:28.157965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk" event={"ID":"c95a5ac1-aeb6-4caf-91b2-986f513c4097","Type":"ContainerStarted","Data":"1b627765bac466beda84fceed97eb0e29d75d5a4974c5ec23e0b3b15710b7e15"}
Sep 15 12:53:32.181068 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:32.181016    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk" event={"ID":"c95a5ac1-aeb6-4caf-91b2-986f513c4097","Type":"ContainerStarted","Data":"f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a"}
Sep 15 12:53:32.181440 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:32.181092    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:35.140465 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:35.140375    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4grfh"
Sep 15 12:53:35.164512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:35.164456    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk" podStartSLOduration=4.18130946 podStartE2EDuration="8.164438502s" totalImagesPullingTime="3.983129042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:53:32.222633347 +0000 UTC m=+530.015335077" watchObservedRunningTime="2026-09-15 12:53:35.164438502 +0000 UTC m=+532.957140233"
Sep 15 12:53:43.188325 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:43.188291    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:45.146638 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.146603    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"]
Sep 15 12:53:45.154835 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.154804    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:45.168110 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.168082    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"]
Sep 15 12:53:45.251684 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.251651    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"]
Sep 15 12:53:45.251982 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.251960    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-zhkmr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl" podUID="d5a594b3-b0cf-432d-b691-c0286fcef632"
Sep 15 12:53:45.282942 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.282903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhkmr\" (UniqueName: \"kubernetes.io/projected/d5a594b3-b0cf-432d-b691-c0286fcef632-kube-api-access-zhkmr\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" (UID: \"d5a594b3-b0cf-432d-b691-c0286fcef632\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:45.283151 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.282952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5a594b3-b0cf-432d-b691-c0286fcef632-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" (UID: \"d5a594b3-b0cf-432d-b691-c0286fcef632\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:45.295005 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.294973    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"]
Sep 15 12:53:45.297743 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.297725    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:45.315438 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.315408    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"]
Sep 15 12:53:45.315711 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.315692    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-kbsnd], unattached volumes=[], failed to process volumes=[extensions-socket-volume kube-api-access-kbsnd]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b"
Sep 15 12:53:45.323358 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.323328    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"]
Sep 15 12:53:45.325043 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.324992    2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"0a6e6321-15ba-4a72-a61b-051df3d0118b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-15T12:53:45Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-15T12:53:45Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-15T12:53:45Z\\\",\\\"message\\\":\\\"containers with unready status: [manager]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-15T12:53:45Z\\\",\\\"message\\\":\\\"containers with unready status: [manager]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"manager\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/tmp/kuadrant\\\",\\\"name\\\":\\\"extensions-socket-volume\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-kbsnd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.132.169\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.132.169\\\"}],\\\"observedGeneration\\\":1,\\\"startTime\\\":\\\"2026-09-15T12:53:45Z\\\"}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-5769f99d9f-42z6s\": pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:45.333338 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.333308    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"]
Sep 15 12:53:45.333534 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.333499    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" containerName="manager" containerID="cri-o://f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a" gracePeriod=2
Sep 15 12:53:45.346169 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.346135    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"]
Sep 15 12:53:45.354047 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.354001    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"]
Sep 15 12:53:45.354628 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.354604    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" containerName="manager"
Sep 15 12:53:45.354628 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.354630    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" containerName="manager"
Sep 15 12:53:45.354802 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.354751    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" containerName="manager"
Sep 15 12:53:45.355806 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.355772    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:45.356748 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.356727    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:45.357710 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.357690    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"]
Sep 15 12:53:45.360812 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.360780    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:45.363825 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.363793    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:45.370387 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.370355    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"]
Sep 15 12:53:45.376594 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.376562    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"]
Sep 15 12:53:45.379374 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.379340    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:45.380250 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.380223    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"]
Sep 15 12:53:45.383732 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.383703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5a594b3-b0cf-432d-b691-c0286fcef632-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" (UID: \"d5a594b3-b0cf-432d-b691-c0286fcef632\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:45.383897 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.383874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhkmr\" (UniqueName: \"kubernetes.io/projected/d5a594b3-b0cf-432d-b691-c0286fcef632-kube-api-access-zhkmr\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" (UID: \"d5a594b3-b0cf-432d-b691-c0286fcef632\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:45.384240 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.384219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5a594b3-b0cf-432d-b691-c0286fcef632-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" (UID: \"d5a594b3-b0cf-432d-b691-c0286fcef632\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:45.389616 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.389595    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zhkmr for pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl: failed to fetch token: pod "kuadrant-operator-controller-manager-7978cbdfbc-mscxl" not found
Sep 15 12:53:45.389712 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.389668    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5a594b3-b0cf-432d-b691-c0286fcef632-kube-api-access-zhkmr podName:d5a594b3-b0cf-432d-b691-c0286fcef632 nodeName:}" failed. No retries permitted until 2026-09-15 12:53:45.889650823 +0000 UTC m=+543.682352532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zhkmr" (UniqueName: "kubernetes.io/projected/d5a594b3-b0cf-432d-b691-c0286fcef632-kube-api-access-zhkmr") pod "kuadrant-operator-controller-manager-7978cbdfbc-mscxl" (UID: "d5a594b3-b0cf-432d-b691-c0286fcef632") : failed to fetch token: pod "kuadrant-operator-controller-manager-7978cbdfbc-mscxl" not found
Sep 15 12:53:45.403071 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.402970    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:45.406298 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.406259    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:45.407086 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.407060    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"]
Sep 15 12:53:45.484381 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.484346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e438ef9-f5c6-46f1-83e5-433705fc23f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lr8hn\" (UID: \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:45.484381 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.484385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fbsm\" (UniqueName: \"kubernetes.io/projected/d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5-kube-api-access-7fbsm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9mx7j\" (UID: \"d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:45.484582 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.484412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cppr6\" (UniqueName: \"kubernetes.io/projected/4e438ef9-f5c6-46f1-83e5-433705fc23f4-kube-api-access-cppr6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lr8hn\" (UID: \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:45.484582 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.484563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9mx7j\" (UID: \"d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:45.572623 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.572603    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:45.575171 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.575144    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:45.577317 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.577296    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:45.585763 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.585740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9mx7j\" (UID: \"d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:45.585811 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.585789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e438ef9-f5c6-46f1-83e5-433705fc23f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lr8hn\" (UID: \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:45.585851 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.585809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fbsm\" (UniqueName: \"kubernetes.io/projected/d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5-kube-api-access-7fbsm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9mx7j\" (UID: \"d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:45.585851 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.585842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cppr6\" (UniqueName: \"kubernetes.io/projected/4e438ef9-f5c6-46f1-83e5-433705fc23f4-kube-api-access-cppr6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lr8hn\" (UID: \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:45.586184 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.586162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9mx7j\" (UID: \"d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:45.586262 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.586180    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e438ef9-f5c6-46f1-83e5-433705fc23f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lr8hn\" (UID: \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:45.599340 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.599317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cppr6\" (UniqueName: \"kubernetes.io/projected/4e438ef9-f5c6-46f1-83e5-433705fc23f4-kube-api-access-cppr6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lr8hn\" (UID: \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:45.599426 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.599394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fbsm\" (UniqueName: \"kubernetes.io/projected/d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5-kube-api-access-7fbsm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-9mx7j\" (UID: \"d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:45.686955 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.686930    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds67v\" (UniqueName: \"kubernetes.io/projected/c95a5ac1-aeb6-4caf-91b2-986f513c4097-kube-api-access-ds67v\") pod \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\" (UID: \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\") "
Sep 15 12:53:45.687098 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.687021    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c95a5ac1-aeb6-4caf-91b2-986f513c4097-extensions-socket-volume\") pod \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\" (UID: \"c95a5ac1-aeb6-4caf-91b2-986f513c4097\") "
Sep 15 12:53:45.687370 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.687345    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95a5ac1-aeb6-4caf-91b2-986f513c4097-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c95a5ac1-aeb6-4caf-91b2-986f513c4097" (UID: "c95a5ac1-aeb6-4caf-91b2-986f513c4097"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:45.689066 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.689044    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95a5ac1-aeb6-4caf-91b2-986f513c4097-kube-api-access-ds67v" (OuterVolumeSpecName: "kube-api-access-ds67v") pod "c95a5ac1-aeb6-4caf-91b2-986f513c4097" (UID: "c95a5ac1-aeb6-4caf-91b2-986f513c4097"). InnerVolumeSpecName "kube-api-access-ds67v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:53:45.709750 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.709728    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:45.717776 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.717758    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:45.789105 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.789061    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds67v\" (UniqueName: \"kubernetes.io/projected/c95a5ac1-aeb6-4caf-91b2-986f513c4097-kube-api-access-ds67v\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:53:45.789105 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.789085    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c95a5ac1-aeb6-4caf-91b2-986f513c4097-extensions-socket-volume\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:53:45.889864 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.889822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhkmr\" (UniqueName: \"kubernetes.io/projected/d5a594b3-b0cf-432d-b691-c0286fcef632-kube-api-access-zhkmr\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" (UID: \"d5a594b3-b0cf-432d-b691-c0286fcef632\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:45.892804 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.892785    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zhkmr for pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl: failed to fetch token: pod "kuadrant-operator-controller-manager-7978cbdfbc-mscxl" not found
Sep 15 12:53:45.892877 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:45.892847    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5a594b3-b0cf-432d-b691-c0286fcef632-kube-api-access-zhkmr podName:d5a594b3-b0cf-432d-b691-c0286fcef632 nodeName:}" failed. No retries permitted until 2026-09-15 12:53:46.892832149 +0000 UTC m=+544.685533862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zhkmr" (UniqueName: "kubernetes.io/projected/d5a594b3-b0cf-432d-b691-c0286fcef632-kube-api-access-zhkmr") pod "kuadrant-operator-controller-manager-7978cbdfbc-mscxl" (UID: "d5a594b3-b0cf-432d-b691-c0286fcef632") : failed to fetch token: pod "kuadrant-operator-controller-manager-7978cbdfbc-mscxl" not found
Sep 15 12:53:45.996429 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.996339    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"]
Sep 15 12:53:45.999962 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:45.999936    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"]
Sep 15 12:53:46.000717 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:53:46.000693    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ca137f_8d7b_4dc1_beac_fd2c4ebb4ae5.slice/crio-34e7d8f8cfdfa4c11e2fdeac3a8e283e3b5ec7a8b7889b40022138fb6ee2dd3c WatchSource:0}: Error finding container 34e7d8f8cfdfa4c11e2fdeac3a8e283e3b5ec7a8b7889b40022138fb6ee2dd3c: Status 404 returned error can't find the container with id 34e7d8f8cfdfa4c11e2fdeac3a8e283e3b5ec7a8b7889b40022138fb6ee2dd3c
Sep 15 12:53:46.006255 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:53:46.006232    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e438ef9_f5c6_46f1_83e5_433705fc23f4.slice/crio-269e7601100d525b1fb77248b7a4263c7a11c5bcabb7e8988e133ca007a75ddc WatchSource:0}: Error finding container 269e7601100d525b1fb77248b7a4263c7a11c5bcabb7e8988e133ca007a75ddc: Status 404 returned error can't find the container with id 269e7601100d525b1fb77248b7a4263c7a11c5bcabb7e8988e133ca007a75ddc
Sep 15 12:53:46.253276 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.253184    2562 generic.go:356] "Generic (PLEG): container finished" podID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" containerID="f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a" exitCode=0
Sep 15 12:53:46.253276 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.253234    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:46.253731 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.253286    2562 scope.go:117] "RemoveContainer" containerID="f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a"
Sep 15 12:53:46.255292 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.255267    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn" event={"ID":"4e438ef9-f5c6-46f1-83e5-433705fc23f4","Type":"ContainerStarted","Data":"288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde"}
Sep 15 12:53:46.255423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.255309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn" event={"ID":"4e438ef9-f5c6-46f1-83e5-433705fc23f4","Type":"ContainerStarted","Data":"269e7601100d525b1fb77248b7a4263c7a11c5bcabb7e8988e133ca007a75ddc"}
Sep 15 12:53:46.255423 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.255364    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:46.255919 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.255891    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:46.257351 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.257323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j" event={"ID":"d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5","Type":"ContainerStarted","Data":"319fb5d55eadd5964455b5296cd564c2a0ef43e1e238f1bc11af7de8ee869328"}
Sep 15 12:53:46.257464 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.257352    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:46.257464 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.257376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j" event={"ID":"d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5","Type":"ContainerStarted","Data":"34e7d8f8cfdfa4c11e2fdeac3a8e283e3b5ec7a8b7889b40022138fb6ee2dd3c"}
Sep 15 12:53:46.257568 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.257486    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:46.257624 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.257553    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:46.257848 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.257828    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:46.259838 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.259815    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="d5a594b3-b0cf-432d-b691-c0286fcef632" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:46.261809 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.261783    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:46.262952 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.262934    2562 scope.go:117] "RemoveContainer" containerID="f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a"
Sep 15 12:53:46.263402 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.263381    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a\": container with ID starting with f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a not found: ID does not exist" containerID="f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a"
Sep 15 12:53:46.263502 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.263411    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a"} err="failed to get container status \"f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a\": rpc error: code = NotFound desc = could not find container \"f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a\": container with ID starting with f2aa0275dca289447cca4fa29293b56c0fc870a49a70b2922288980607d9044a not found: ID does not exist"
Sep 15 12:53:46.263837 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.263816    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:46.266308 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.266292    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:46.269123 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.269107    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:46.289424 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.289375    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn" podStartSLOduration=1.289359424 podStartE2EDuration="1.289359424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:53:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:53:46.288677268 +0000 UTC m=+544.081379036" watchObservedRunningTime="2026-09-15 12:53:46.289359424 +0000 UTC m=+544.082061154"
Sep 15 12:53:46.315361 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.315307    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j" podStartSLOduration=1.315288964 podStartE2EDuration="1.315288964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:53:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:53:46.314628324 +0000 UTC m=+544.107330069" watchObservedRunningTime="2026-09-15 12:53:46.315288964 +0000 UTC m=+544.107990694"
Sep 15 12:53:46.317022 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.316990    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="d5a594b3-b0cf-432d-b691-c0286fcef632" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:46.319648 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.319622    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-6rfnk\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-6rfnk"
Sep 15 12:53:46.322205 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:46.322171    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:46.394680 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.394648    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5a594b3-b0cf-432d-b691-c0286fcef632-extensions-socket-volume\") pod \"d5a594b3-b0cf-432d-b691-c0286fcef632\" (UID: \"d5a594b3-b0cf-432d-b691-c0286fcef632\") "
Sep 15 12:53:46.394914 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.394889    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a594b3-b0cf-432d-b691-c0286fcef632-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d5a594b3-b0cf-432d-b691-c0286fcef632" (UID: "d5a594b3-b0cf-432d-b691-c0286fcef632"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:46.395102 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.395073    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhkmr\" (UniqueName: \"kubernetes.io/projected/d5a594b3-b0cf-432d-b691-c0286fcef632-kube-api-access-zhkmr\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:53:46.395102 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.395098    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5a594b3-b0cf-432d-b691-c0286fcef632-extensions-socket-volume\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:53:46.756636 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.756604    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" path="/var/lib/kubelet/pods/0a6e6321-15ba-4a72-a61b-051df3d0118b/volumes"
Sep 15 12:53:46.756847 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.756833    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c95a5ac1-aeb6-4caf-91b2-986f513c4097" path="/var/lib/kubelet/pods/c95a5ac1-aeb6-4caf-91b2-986f513c4097/volumes"
Sep 15 12:53:46.757151 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:46.757138    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5a594b3-b0cf-432d-b691-c0286fcef632" path="/var/lib/kubelet/pods/d5a594b3-b0cf-432d-b691-c0286fcef632/volumes"
Sep 15 12:53:47.269374 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:47.269346    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:47.269849 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:47.269346    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:47.271994 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:47.271967    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:47.274346 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:47.274320    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:47.276812 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:47.276790    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-mscxl\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="d5a594b3-b0cf-432d-b691-c0286fcef632" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-mscxl"
Sep 15 12:53:47.279247 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:47.279208    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-42z6s\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0a6e6321-15ba-4a72-a61b-051df3d0118b" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-42z6s"
Sep 15 12:53:57.271823 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.271792    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-9mx7j"
Sep 15 12:53:57.272281 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.271889    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:57.367552 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.367521    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"]
Sep 15 12:53:57.367745 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.367700    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn" podUID="4e438ef9-f5c6-46f1-83e5-433705fc23f4" containerName="manager" containerID="cri-o://288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde" gracePeriod=10
Sep 15 12:53:57.649766 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.649734    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:57.803308 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.803279    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cppr6\" (UniqueName: \"kubernetes.io/projected/4e438ef9-f5c6-46f1-83e5-433705fc23f4-kube-api-access-cppr6\") pod \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\" (UID: \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\") "
Sep 15 12:53:57.803480 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.803343    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e438ef9-f5c6-46f1-83e5-433705fc23f4-extensions-socket-volume\") pod \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\" (UID: \"4e438ef9-f5c6-46f1-83e5-433705fc23f4\") "
Sep 15 12:53:57.803720 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.803694    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e438ef9-f5c6-46f1-83e5-433705fc23f4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4e438ef9-f5c6-46f1-83e5-433705fc23f4" (UID: "4e438ef9-f5c6-46f1-83e5-433705fc23f4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:53:57.805474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.805454    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e438ef9-f5c6-46f1-83e5-433705fc23f4-kube-api-access-cppr6" (OuterVolumeSpecName: "kube-api-access-cppr6") pod "4e438ef9-f5c6-46f1-83e5-433705fc23f4" (UID: "4e438ef9-f5c6-46f1-83e5-433705fc23f4"). InnerVolumeSpecName "kube-api-access-cppr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:53:57.904891 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.904850    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cppr6\" (UniqueName: \"kubernetes.io/projected/4e438ef9-f5c6-46f1-83e5-433705fc23f4-kube-api-access-cppr6\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:53:57.904891 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:57.904883    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e438ef9-f5c6-46f1-83e5-433705fc23f4-extensions-socket-volume\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:53:58.324869 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.324835    2562 generic.go:356] "Generic (PLEG): container finished" podID="4e438ef9-f5c6-46f1-83e5-433705fc23f4" containerID="288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde" exitCode=0
Sep 15 12:53:58.325356 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.324880    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"
Sep 15 12:53:58.325356 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.324921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn" event={"ID":"4e438ef9-f5c6-46f1-83e5-433705fc23f4","Type":"ContainerDied","Data":"288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde"}
Sep 15 12:53:58.325356 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.324979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn" event={"ID":"4e438ef9-f5c6-46f1-83e5-433705fc23f4","Type":"ContainerDied","Data":"269e7601100d525b1fb77248b7a4263c7a11c5bcabb7e8988e133ca007a75ddc"}
Sep 15 12:53:58.325356 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.325012    2562 scope.go:117] "RemoveContainer" containerID="288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde"
Sep 15 12:53:58.334286 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.334271    2562 scope.go:117] "RemoveContainer" containerID="288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde"
Sep 15 12:53:58.334583 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:53:58.334565    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde\": container with ID starting with 288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde not found: ID does not exist" containerID="288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde"
Sep 15 12:53:58.334635 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.334590    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde"} err="failed to get container status \"288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde\": rpc error: code = NotFound desc = could not find container \"288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde\": container with ID starting with 288ce85766a7bfeadadb794183ce243ae172e9e5412c1b816831a68f0f358dde not found: ID does not exist"
Sep 15 12:53:58.379864 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.379833    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"]
Sep 15 12:53:58.385510 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.385489    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lr8hn"]
Sep 15 12:53:58.755581 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:53:58.755549    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e438ef9-f5c6-46f1-83e5-433705fc23f4" path="/var/lib/kubelet/pods/4e438ef9-f5c6-46f1-83e5-433705fc23f4/volumes"
Sep 15 12:54:13.823498 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.823465    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"]
Sep 15 12:54:13.823960 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.823943    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e438ef9-f5c6-46f1-83e5-433705fc23f4" containerName="manager"
Sep 15 12:54:13.824022 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.823964    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e438ef9-f5c6-46f1-83e5-433705fc23f4" containerName="manager"
Sep 15 12:54:13.824112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.824098    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e438ef9-f5c6-46f1-83e5-433705fc23f4" containerName="manager"
Sep 15 12:54:13.826361 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.826343    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.829197 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.829179    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v9x5z\""
Sep 15 12:54:13.841933 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.841909    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"]
Sep 15 12:54:13.959878 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.959842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.960112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.959890    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.960112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.959912    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.960112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.959970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d584efee-f195-4b32-90a9-a158bb292b71-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.960112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.960011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d584efee-f195-4b32-90a9-a158bb292b71-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.960112 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.960099    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vgc6\" (UniqueName: \"kubernetes.io/projected/d584efee-f195-4b32-90a9-a158bb292b71-kube-api-access-5vgc6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.960370 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.960191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.960370 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.960215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d584efee-f195-4b32-90a9-a158bb292b71-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:13.960370 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:13.960243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.061678 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vgc6\" (UniqueName: \"kubernetes.io/projected/d584efee-f195-4b32-90a9-a158bb292b71-kube-api-access-5vgc6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.061876 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.061876 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d584efee-f195-4b32-90a9-a158bb292b71-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.061876 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.061876 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.061876 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061837    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.061876 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061864    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.062234 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061888    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d584efee-f195-4b32-90a9-a158bb292b71-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.062234 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.061912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d584efee-f195-4b32-90a9-a158bb292b71-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.062234 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.062159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.062469 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.062365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.062469 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.062413    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.062576 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.062479    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.062729 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.062709    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d584efee-f195-4b32-90a9-a158bb292b71-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.064147 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.064119    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d584efee-f195-4b32-90a9-a158bb292b71-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.064414 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.064397    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d584efee-f195-4b32-90a9-a158bb292b71-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.072460 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.072434    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d584efee-f195-4b32-90a9-a158bb292b71-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.072460 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.072450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vgc6\" (UniqueName: \"kubernetes.io/projected/d584efee-f195-4b32-90a9-a158bb292b71-kube-api-access-5vgc6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8jxq4\" (UID: \"d584efee-f195-4b32-90a9-a158bb292b71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.136795 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.136706    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:14.415875 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.415789    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"]
Sep 15 12:54:14.419480 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:54:14.419454    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd584efee_f195_4b32_90a9_a158bb292b71.slice/crio-f52554f64db5fea797dfd43aad6317276a0276e11ff7c873868d1070881b6dfa WatchSource:0}: Error finding container f52554f64db5fea797dfd43aad6317276a0276e11ff7c873868d1070881b6dfa: Status 404 returned error can't find the container with id f52554f64db5fea797dfd43aad6317276a0276e11ff7c873868d1070881b6dfa
Sep 15 12:54:14.421736 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.421700    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:54:14.421826 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.421786    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:54:14.421865 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:14.421828    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:54:15.413697 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:15.413664    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" event={"ID":"d584efee-f195-4b32-90a9-a158bb292b71","Type":"ContainerStarted","Data":"420498f3bdd8120da25948911532f021cf1c0104885ee142dbb6a8bf1389c055"}
Sep 15 12:54:15.414118 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:15.413713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" event={"ID":"d584efee-f195-4b32-90a9-a158bb292b71","Type":"ContainerStarted","Data":"f52554f64db5fea797dfd43aad6317276a0276e11ff7c873868d1070881b6dfa"}
Sep 15 12:54:15.443499 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:15.443447    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" podStartSLOduration=2.443428183 podStartE2EDuration="2.443428183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:54:15.44225097 +0000 UTC m=+573.234952702" watchObservedRunningTime="2026-09-15 12:54:15.443428183 +0000 UTC m=+573.236129970"
Sep 15 12:54:16.136919 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.136885    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:16.141504 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.141480    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:16.419345 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.419265    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:16.420327 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.420307    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:54:16.823874 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.823837    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:54:16.826800 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.826776    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:16.829119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.829097    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 15 12:54:16.829264 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.829247    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-w5ghz\""
Sep 15 12:54:16.844657 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.844623    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:54:16.857783 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.857749    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:54:16.888628 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.888593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9j29\" (UniqueName: \"kubernetes.io/projected/2de15519-4f94-4f66-abb2-1fead0097a15-kube-api-access-z9j29\") pod \"limitador-limitador-84994f7ffc-92knm\" (UID: \"2de15519-4f94-4f66-abb2-1fead0097a15\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:16.888832 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.888716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2de15519-4f94-4f66-abb2-1fead0097a15-config-file\") pod \"limitador-limitador-84994f7ffc-92knm\" (UID: \"2de15519-4f94-4f66-abb2-1fead0097a15\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:16.990125 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.990086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9j29\" (UniqueName: \"kubernetes.io/projected/2de15519-4f94-4f66-abb2-1fead0097a15-kube-api-access-z9j29\") pod \"limitador-limitador-84994f7ffc-92knm\" (UID: \"2de15519-4f94-4f66-abb2-1fead0097a15\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:16.990334 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.990181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2de15519-4f94-4f66-abb2-1fead0097a15-config-file\") pod \"limitador-limitador-84994f7ffc-92knm\" (UID: \"2de15519-4f94-4f66-abb2-1fead0097a15\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:16.990775 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.990756    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2de15519-4f94-4f66-abb2-1fead0097a15-config-file\") pod \"limitador-limitador-84994f7ffc-92knm\" (UID: \"2de15519-4f94-4f66-abb2-1fead0097a15\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:16.998338 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:16.998313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9j29\" (UniqueName: \"kubernetes.io/projected/2de15519-4f94-4f66-abb2-1fead0097a15-kube-api-access-z9j29\") pod \"limitador-limitador-84994f7ffc-92knm\" (UID: \"2de15519-4f94-4f66-abb2-1fead0097a15\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:17.137805 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:17.137732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:17.420102 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:17.420002    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:54:17.423182 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:54:17.423148    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de15519_4f94_4f66_abb2_1fead0097a15.slice/crio-0d9154048fb8ee44652dbe345fcaddb4ae781e7f8fc6158b08a09f39adda1496 WatchSource:0}: Error finding container 0d9154048fb8ee44652dbe345fcaddb4ae781e7f8fc6158b08a09f39adda1496: Status 404 returned error can't find the container with id 0d9154048fb8ee44652dbe345fcaddb4ae781e7f8fc6158b08a09f39adda1496
Sep 15 12:54:18.432041 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:18.432006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm" event={"ID":"2de15519-4f94-4f66-abb2-1fead0097a15","Type":"ContainerStarted","Data":"5cd02eea9305fc6ed6c41308447c559ae0501d49c739b0726005573155f0c0f2"}
Sep 15 12:54:18.432358 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:18.432101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm" event={"ID":"2de15519-4f94-4f66-abb2-1fead0097a15","Type":"ContainerStarted","Data":"0d9154048fb8ee44652dbe345fcaddb4ae781e7f8fc6158b08a09f39adda1496"}
Sep 15 12:54:18.454718 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:18.454676    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm" podStartSLOduration=1.559254355 podStartE2EDuration="2.454662826s" totalImagesPullingTime="895.408471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:54:18.451820309 +0000 UTC m=+576.244522040" watchObservedRunningTime="2026-09-15 12:54:18.454662826 +0000 UTC m=+576.247364619"
Sep 15 12:54:19.437005 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:19.436969    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:30.442985 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:30.442951    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-92knm"
Sep 15 12:54:42.667759 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:42.667720    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:54:42.668247 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:42.668051    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:54:42.671636 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:42.671609    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:54:42.671800 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:54:42.671751    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:55:29.213234 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:29.213189    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:55:38.827516 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:38.827484    2562 generic.go:356] "Generic (PLEG): container finished" podID="d584efee-f195-4b32-90a9-a158bb292b71" containerID="420498f3bdd8120da25948911532f021cf1c0104885ee142dbb6a8bf1389c055" exitCode=0
Sep 15 12:55:38.827910 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:38.827528    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" event={"ID":"d584efee-f195-4b32-90a9-a158bb292b71","Type":"ContainerDied","Data":"420498f3bdd8120da25948911532f021cf1c0104885ee142dbb6a8bf1389c055"}
Sep 15 12:55:38.827910 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:38.827866    2562 scope.go:117] "RemoveContainer" containerID="420498f3bdd8120da25948911532f021cf1c0104885ee142dbb6a8bf1389c055"
Sep 15 12:55:38.828935 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:38.828904    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:55:38.828989 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:38.828980    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:55:38.829023 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:38.829013    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:55:39.838270 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:39.838228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" event={"ID":"d584efee-f195-4b32-90a9-a158bb292b71","Type":"ContainerStarted","Data":"5302cb414d36dead376a95af86f36aa516042d5cc48c0ba985e96e3bc12d89b1"}
Sep 15 12:55:40.137348 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:40.137246    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:55:40.137535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:40.137385    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8jxq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 15 12:55:40.137535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:40.137436    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" podUID="d584efee-f195-4b32-90a9-a158bb292b71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 15 12:55:41.137364 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:41.137324    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8jxq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 15 12:55:41.137851 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:41.137409    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" podUID="d584efee-f195-4b32-90a9-a158bb292b71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 15 12:55:42.137480 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:42.137440    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8jxq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 15 12:55:42.138087 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:42.138055    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" podUID="d584efee-f195-4b32-90a9-a158bb292b71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 15 12:55:43.137172 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:43.137131    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8jxq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 15 12:55:43.137378 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:43.137208    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" podUID="d584efee-f195-4b32-90a9-a158bb292b71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 15 12:55:44.139405 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:44.139367    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8jxq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 15 12:55:44.139964 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:44.139933    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4" podUID="d584efee-f195-4b32-90a9-a158bb292b71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 15 12:55:45.141243 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:45.141211    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:55:45.141608 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:45.141502    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:55:45.142101 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:45.142084    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8jxq4"
Sep 15 12:55:51.826093 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:55:51.826061    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:56:00.778931 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.778892    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"]
Sep 15 12:56:00.782454 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.782433    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.785295 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.785270    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 12:56:00.786361 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.786337    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6p47f\""
Sep 15 12:56:00.786470 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.786455    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 15 12:56:00.786611 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.786597    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 12:56:00.795879 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.795856    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"]
Sep 15 12:56:00.856254 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.856221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.856415 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.856260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjjxm\" (UniqueName: \"kubernetes.io/projected/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-kube-api-access-rjjxm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.856415 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.856302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.856415 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.856345    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.856415 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.856366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.856415 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.856383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.957499 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.957461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.957690 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.957508    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjjxm\" (UniqueName: \"kubernetes.io/projected/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-kube-api-access-rjjxm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.957690 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.957552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.957690 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.957595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.957690 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.957630    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.957690 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.957659    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.957948 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.957917    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.958086 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.958057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.958175 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.958107    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.959837 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.959810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.960109 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.960088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:00.974651 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:00.974618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjjxm\" (UniqueName: \"kubernetes.io/projected/034b91e5-25f7-4b7b-a30b-1e5d108ba40d-kube-api-access-rjjxm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxtjk\" (UID: \"034b91e5-25f7-4b7b-a30b-1e5d108ba40d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:01.092799 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:01.092709    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:01.367942 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:01.367862    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"]
Sep 15 12:56:01.370995 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:56:01.370966    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034b91e5_25f7_4b7b_a30b_1e5d108ba40d.slice/crio-3a223ab262c5f7e079b826d2e9e1de5841e7cd09f1885ffce5a8334554b2c65f WatchSource:0}: Error finding container 3a223ab262c5f7e079b826d2e9e1de5841e7cd09f1885ffce5a8334554b2c65f: Status 404 returned error can't find the container with id 3a223ab262c5f7e079b826d2e9e1de5841e7cd09f1885ffce5a8334554b2c65f
Sep 15 12:56:01.609885 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:01.609851    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:56:01.949587 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:01.949547    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk" event={"ID":"034b91e5-25f7-4b7b-a30b-1e5d108ba40d","Type":"ContainerStarted","Data":"3a223ab262c5f7e079b826d2e9e1de5841e7cd09f1885ffce5a8334554b2c65f"}
Sep 15 12:56:02.955658 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:02.955616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk" event={"ID":"034b91e5-25f7-4b7b-a30b-1e5d108ba40d","Type":"ContainerStarted","Data":"15637ee8aa1e34a6171cc0fc8872b421e9365f8bb96249fe3399654ae5dcca94"}
Sep 15 12:56:02.976815 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:02.976770    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk" podStartSLOduration=2.01404468 podStartE2EDuration="2.976755157s" totalImagesPullingTime="962.710477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:56:02.974549345 +0000 UTC m=+680.767251077" watchObservedRunningTime="2026-09-15 12:56:02.976755157 +0000 UTC m=+680.769456888"
Sep 15 12:56:08.987975 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:08.987946    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:56:11.093248 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:11.093212    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:11.093630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:11.093280    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:11.105514 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:11.105490    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:12.014190 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:12.014163    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxtjk"
Sep 15 12:56:12.286839 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:12.286755    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:56:23.880441 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.880404    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"]
Sep 15 12:56:23.883643 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.883622    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:23.886337 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.886317    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 15 12:56:23.895013 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.894990    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"]
Sep 15 12:56:23.973364 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.973324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:23.973364 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.973368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:23.973559 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.973392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ecc810-f364-4665-8791-9d42635c59cb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:23.973559 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.973491    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gkzd\" (UniqueName: \"kubernetes.io/projected/92ecc810-f364-4665-8791-9d42635c59cb-kube-api-access-5gkzd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:23.973635 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.973584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:23.973669 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:23.973649    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.074851 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.074818    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.075058 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.074861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.075058 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.074889    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.075058 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.074915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ecc810-f364-4665-8791-9d42635c59cb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.075058 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.074958    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gkzd\" (UniqueName: \"kubernetes.io/projected/92ecc810-f364-4665-8791-9d42635c59cb-kube-api-access-5gkzd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.075058 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.075054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.075343 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.075318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.075399 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.075367    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.075713 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.075692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.077268 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.077248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ecc810-f364-4665-8791-9d42635c59cb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.077485 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.077468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ecc810-f364-4665-8791-9d42635c59cb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.083455 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.083432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gkzd\" (UniqueName: \"kubernetes.io/projected/92ecc810-f364-4665-8791-9d42635c59cb-kube-api-access-5gkzd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4\" (UID: \"92ecc810-f364-4665-8791-9d42635c59cb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.193560 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.193530    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:24.470478 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:24.470448    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"]
Sep 15 12:56:24.472156 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:56:24.472123    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ecc810_f364_4665_8791_9d42635c59cb.slice/crio-fa578d393b52a1a678851d1f25dbdd10d69e409b90e0cbdbada607c3e83b69ab WatchSource:0}: Error finding container fa578d393b52a1a678851d1f25dbdd10d69e409b90e0cbdbada607c3e83b69ab: Status 404 returned error can't find the container with id fa578d393b52a1a678851d1f25dbdd10d69e409b90e0cbdbada607c3e83b69ab
Sep 15 12:56:25.076007 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:25.075971    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4" event={"ID":"92ecc810-f364-4665-8791-9d42635c59cb","Type":"ContainerStarted","Data":"8920f728e21c3d18f42168f97498c78a018f356944afeaab5d2a9e4fb894d528"}
Sep 15 12:56:25.076375 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:25.076025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4" event={"ID":"92ecc810-f364-4665-8791-9d42635c59cb","Type":"ContainerStarted","Data":"fa578d393b52a1a678851d1f25dbdd10d69e409b90e0cbdbada607c3e83b69ab"}
Sep 15 12:56:25.100837 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:25.100776    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4" podStartSLOduration=1.935053643 podStartE2EDuration="2.100755859s" totalImagesPullingTime="165.702216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:56:25.100462967 +0000 UTC m=+702.893164699" watchObservedRunningTime="2026-09-15 12:56:25.100755859 +0000 UTC m=+702.893457591"
Sep 15 12:56:25.643669 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:25.643633    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:56:31.082783 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:31.082747    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:56:34.194354 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:34.194317    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:34.194822 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:34.194409    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:34.206862 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:34.206834    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:35.136582 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:35.136496    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4"
Sep 15 12:56:44.392361 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:56:44.392325    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:58:13.949048 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:13.949004    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"]
Sep 15 12:58:13.952707 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:13.952684    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:13.972340 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:13.972315    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-2af9a3-openshift-default-dockercfg-vzxp6\""
Sep 15 12:58:13.975507 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:13.975476    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"]
Sep 15 12:58:14.006534 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006499    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-data\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.006709 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006559    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clgj4\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-kube-api-access-clgj4\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.006709 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006625    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-podinfo\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.006709 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006655    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-token\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.007122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006825    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istiod-ca-cert\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.007122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-credential-socket\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.007122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-envoy\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.007122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-certs\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.007122 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.006978    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-socket\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.016335 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.016307    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"]
Sep 15 12:58:14.019963 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.019939    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.022837 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.022812    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-6c9f0b-openshift-default-dockercfg-x59mq\""
Sep 15 12:58:14.037459 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.037436    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"]
Sep 15 12:58:14.108114 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108084    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58dhb\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-kube-api-access-58dhb\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.108291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istiod-ca-cert\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.108291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-socket\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.108291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-credential-socket\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.108291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108237    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-data\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.108535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108298    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-credential-socket\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.108535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108317    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"]
Sep 15 12:58:14.108535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108330    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-envoy\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.108535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108360    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-certs\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.108535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-socket\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.108790 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-credential-socket\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.108790 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-data\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.108790 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108737    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-envoy\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.108790 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-podinfo\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.109010 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-socket\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.109010 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clgj4\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-kube-api-access-clgj4\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.109010 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.108704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-certs\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.109184 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.109090    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-data\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.109184 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.109088    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-podinfo\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.109184 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.109142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-certs\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.109184 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.109166    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istiod-ca-cert\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.109184 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.109175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-token\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.109435 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.109316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c527b9ee-4b9b-489e-b168-d985cbf47219-istiod-ca-cert\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.109435 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.109354    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-token\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.110987 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.110965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-envoy\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.111475 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.111459    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-podinfo\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.112705 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.112691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.115997 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.115978    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-528f6b-openshift-default-dockercfg-2bxpn\""
Sep 15 12:58:14.122220 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.122199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-token\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.123502 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.123481    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clgj4\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-kube-api-access-clgj4\") pod \"e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") " pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.128253 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.128233    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"]
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58dhb\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-kube-api-access-58dhb\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210744    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-socket\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210778    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlbq5\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-kube-api-access-nlbq5\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210807    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-socket\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210840    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-credential-socket\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-data\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-certs\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-credential-socket\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.210997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-envoy\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-podinfo\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211110    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istiod-ca-cert\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211148    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-envoy\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-data\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.213630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-certs\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-token\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-podinfo\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c527b9ee-4b9b-489e-b168-d985cbf47219-istiod-ca-cert\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.211327    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-token\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.212260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-socket\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.212508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-credential-socket\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.212742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-data\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.213152    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-certs\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.214472 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.213476    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c527b9ee-4b9b-489e-b168-d985cbf47219-istiod-ca-cert\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.215818 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.215794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-envoy\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.215818 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.215807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-podinfo\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.228573 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.228542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58dhb\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-kube-api-access-58dhb\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.228733 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.228545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-token\") pod \"e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") " pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.262922 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.262889    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:14.312294 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istiod-ca-cert\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312294 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-envoy\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312294 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-data\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312294 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312268    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-token\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312294 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-podinfo\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312589 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlbq5\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-kube-api-access-nlbq5\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312589 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-socket\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312589 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-certs\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312589 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312517    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-credential-socket\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.312959 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.312923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-credential-socket\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.313123 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.313094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-data\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.313387 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.313360    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-socket\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.313520 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.313426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istiod-ca-cert\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.313520 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.313469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-certs\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.315943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.315897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-podinfo\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.316801 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.316444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-envoy\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.328754 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.328725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-token\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.332943 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.332688    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:14.333136 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.333111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlbq5\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-kube-api-access-nlbq5\") pod \"e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") " pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.445948 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.445917    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:14.543490 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.543465    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"]
Sep 15 12:58:14.545174 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:58:14.545145    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec96a3c_240c_476b_ac5c_a3d0fe8b1ef8.slice/crio-ee7db6ece36a79f1b91196d010ba41a278690d515b3691864180fc701ccf8031 WatchSource:0}: Error finding container ee7db6ece36a79f1b91196d010ba41a278690d515b3691864180fc701ccf8031: Status 404 returned error can't find the container with id ee7db6ece36a79f1b91196d010ba41a278690d515b3691864180fc701ccf8031
Sep 15 12:58:14.547264 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.547235    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:14.547330 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.547300    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:14.547369 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.547328    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:14.612886 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.612862    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"]
Sep 15 12:58:14.615105 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:58:14.615083    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc527b9ee_4b9b_489e_b168_d985cbf47219.slice/crio-de81556079892e620cfbfe0d67390afaf07084d62c749d4ecbf0b493c2098e56 WatchSource:0}: Error finding container de81556079892e620cfbfe0d67390afaf07084d62c749d4ecbf0b493c2098e56: Status 404 returned error can't find the container with id de81556079892e620cfbfe0d67390afaf07084d62c749d4ecbf0b493c2098e56
Sep 15 12:58:14.617685 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.617653    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:14.617758 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.617716    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:14.617758 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.617743    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:14.648867 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.648823    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" event={"ID":"c527b9ee-4b9b-489e-b168-d985cbf47219","Type":"ContainerStarted","Data":"de81556079892e620cfbfe0d67390afaf07084d62c749d4ecbf0b493c2098e56"}
Sep 15 12:58:14.653970 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.653938    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" event={"ID":"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8","Type":"ContainerStarted","Data":"cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b"}
Sep 15 12:58:14.654108 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.653991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" event={"ID":"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8","Type":"ContainerStarted","Data":"ee7db6ece36a79f1b91196d010ba41a278690d515b3691864180fc701ccf8031"}
Sep 15 12:58:14.680260 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.680206    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podStartSLOduration=1.680185728 podStartE2EDuration="1.680185728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:58:14.677711721 +0000 UTC m=+812.470413481" watchObservedRunningTime="2026-09-15 12:58:14.680185728 +0000 UTC m=+812.472887460"
Sep 15 12:58:14.730263 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.730213    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"]
Sep 15 12:58:14.735874 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:58:14.735831    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ec43ad_25ce_49f1_9a67_d6c1880c3f9c.slice/crio-f09747bea3bb4b594530c71759bd8bae6242f5edaf497563f59f0ced5181ca77 WatchSource:0}: Error finding container f09747bea3bb4b594530c71759bd8bae6242f5edaf497563f59f0ced5181ca77: Status 404 returned error can't find the container with id f09747bea3bb4b594530c71759bd8bae6242f5edaf497563f59f0ced5181ca77
Sep 15 12:58:14.740629 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.739904    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:14.740629 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.739991    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:14.740629 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:14.740110    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:15.263958 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.263911    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:15.268824 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.268789    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:15.660517 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.660419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" event={"ID":"c527b9ee-4b9b-489e-b168-d985cbf47219","Type":"ContainerStarted","Data":"0cc3e9026233ee8f798df7f436c675c8f66500e08892a0f43a871b65f67968c5"}
Sep 15 12:58:15.661966 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.661935    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" event={"ID":"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c","Type":"ContainerStarted","Data":"354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27"}
Sep 15 12:58:15.662117 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.662059    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:15.662117 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.662102    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" event={"ID":"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c","Type":"ContainerStarted","Data":"f09747bea3bb4b594530c71759bd8bae6242f5edaf497563f59f0ced5181ca77"}
Sep 15 12:58:15.662920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.662901    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:15.682346 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.682303    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podStartSLOduration=2.682289933 podStartE2EDuration="2.682289933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:58:15.681773414 +0000 UTC m=+813.474475201" watchObservedRunningTime="2026-09-15 12:58:15.682289933 +0000 UTC m=+813.474991664"
Sep 15 12:58:15.730729 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:15.730678    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podStartSLOduration=1.7306606900000001 podStartE2EDuration="1.73066069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:58:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:58:15.728207121 +0000 UTC m=+813.520908852" watchObservedRunningTime="2026-09-15 12:58:15.73066069 +0000 UTC m=+813.523362421"
Sep 15 12:58:16.333614 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:16.333571    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:16.338731 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:16.338706    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:16.447134 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:16.447096    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:16.452514 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:16.452487    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:16.669088 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:16.668993    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:16.669337 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:16.669314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:16.669997 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:16.669973    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:16.670234 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:16.670217    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:17.901496 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:17.901460    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:58:23.649637 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.649600    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"]
Sep 15 12:58:23.653142 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.653117    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.656601 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.656580    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-08701e10-openshift-default-dockercfg-k49v8\""
Sep 15 12:58:23.669618 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.669592    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"]
Sep 15 12:58:23.713906 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.713874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-data\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.713906 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.713908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-socket\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.714134 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.713987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-podinfo\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.714134 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.714109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf9f1d07-f711-4d0c-9961-a2823c61b721-istiod-ca-cert\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.714220 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.714135    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-envoy\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.714220 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.714203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-certs\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.714288 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.714233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-token\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.714288 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.714256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lggsf\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-kube-api-access-lggsf\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.714288 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.714286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-credential-socket\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.814873 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.814830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-credential-socket\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815101 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.814894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-data\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815101 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.814923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-socket\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815101 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.814983    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-podinfo\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815371 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf9f1d07-f711-4d0c-9961-a2823c61b721-istiod-ca-cert\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815371 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-envoy\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815371 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-certs\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815371 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815322    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-token\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815371 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-credential-socket\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815371 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815353    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lggsf\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-kube-api-access-lggsf\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815702 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-data\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815702 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-certs\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815702 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815699    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-socket\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.815881 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.815807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf9f1d07-f711-4d0c-9961-a2823c61b721-istiod-ca-cert\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.817348 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.817328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-envoy\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.817687 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.817666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-podinfo\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.823886 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.823859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-token\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.824746 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.824719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lggsf\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-kube-api-access-lggsf\") pod \"e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") " pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:23.965186 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:23.965156    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:24.257008 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:24.256981    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"]
Sep 15 12:58:24.259336 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:58:24.259308    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf9f1d07_f711_4d0c_9961_a2823c61b721.slice/crio-b7e40021f13dcce4129febd417027e44bda17ed5ef511a0e8357335e75aef198 WatchSource:0}: Error finding container b7e40021f13dcce4129febd417027e44bda17ed5ef511a0e8357335e75aef198: Status 404 returned error can't find the container with id b7e40021f13dcce4129febd417027e44bda17ed5ef511a0e8357335e75aef198
Sep 15 12:58:24.261556 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:24.261524    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:24.261672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:24.261611    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:24.261672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:24.261655    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:24.730226 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:24.730184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" event={"ID":"cf9f1d07-f711-4d0c-9961-a2823c61b721","Type":"ContainerStarted","Data":"2b0bc502963b7c5293facba8178b522f79fc83b4cf3e5dd01f9cfff930ae90d3"}
Sep 15 12:58:24.730689 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:24.730286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" event={"ID":"cf9f1d07-f711-4d0c-9961-a2823c61b721","Type":"ContainerStarted","Data":"b7e40021f13dcce4129febd417027e44bda17ed5ef511a0e8357335e75aef198"}
Sep 15 12:58:24.755635 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:24.755568    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" podStartSLOduration=1.7555504960000001 podStartE2EDuration="1.755550496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:58:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:58:24.753402695 +0000 UTC m=+822.546104427" watchObservedRunningTime="2026-09-15 12:58:24.755550496 +0000 UTC m=+822.548252227"
Sep 15 12:58:24.965923 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:24.965884    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:25.971433 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:25.971399    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:26.748055 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.744396    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:26.751960 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.751931    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:26.770465 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.770428    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"]
Sep 15 12:58:26.775343 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.775317    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.780467 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.780439    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-59f4a3-openshift-default-dockercfg-4mzlv\""
Sep 15 12:58:26.783775 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.783754    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"]
Sep 15 12:58:26.852690 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.852652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-socket\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.852938 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.852916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c77919fd-12dc-4604-9e31-f62777722f3b-istiod-ca-cert\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.853091 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.853074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c77919fd-12dc-4604-9e31-f62777722f3b-istio-podinfo\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.853210 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.853193    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-credential-socket\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.854257 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.853372    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-istio-token\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.854477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.854454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-data\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.854672 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.854656    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gxml\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-kube-api-access-6gxml\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.855106 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.855086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-certs\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.855281 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.855264    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-envoy\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gxml\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-kube-api-access-6gxml\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-certs\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-envoy\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-socket\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c77919fd-12dc-4604-9e31-f62777722f3b-istiod-ca-cert\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c77919fd-12dc-4604-9e31-f62777722f3b-istio-podinfo\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-credential-socket\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-istio-token\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958051 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.957986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-data\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.958682 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.958651    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-data\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.959899 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.959862    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-credential-socket\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.963059 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.960515    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-socket\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.963059 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.961055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-certs\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.963059 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.961363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c77919fd-12dc-4604-9e31-f62777722f3b-istiod-ca-cert\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.964255 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.963847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c77919fd-12dc-4604-9e31-f62777722f3b-istio-podinfo\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.964255 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.964102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-envoy\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.969920 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.969889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gxml\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-kube-api-access-6gxml\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:26.970068 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:26.970046    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-istio-token\") pod \"e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") " pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:27.090140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:27.090004    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:27.472887 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:27.472837    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"]
Sep 15 12:58:27.491102 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:58:27.490351    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77919fd_12dc_4604_9e31_f62777722f3b.slice/crio-91fabf6480803fd478c500929dc11d441f15f9d527cec855725fbe7cee21603a WatchSource:0}: Error finding container 91fabf6480803fd478c500929dc11d441f15f9d527cec855725fbe7cee21603a: Status 404 returned error can't find the container with id 91fabf6480803fd478c500929dc11d441f15f9d527cec855725fbe7cee21603a
Sep 15 12:58:27.493078 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:27.492901    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:27.493078 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:27.492978    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:27.493078 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:27.493008    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:27.756302 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:27.756210    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" event={"ID":"c77919fd-12dc-4604-9e31-f62777722f3b","Type":"ContainerStarted","Data":"5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350"}
Sep 15 12:58:27.756590 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:27.756568    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" event={"ID":"c77919fd-12dc-4604-9e31-f62777722f3b","Type":"ContainerStarted","Data":"91fabf6480803fd478c500929dc11d441f15f9d527cec855725fbe7cee21603a"}
Sep 15 12:58:27.781568 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:27.781488    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podStartSLOduration=1.781466684 podStartE2EDuration="1.781466684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:58:27.780193827 +0000 UTC m=+825.572895559" watchObservedRunningTime="2026-09-15 12:58:27.781466684 +0000 UTC m=+825.574168407"
Sep 15 12:58:28.092993 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:28.092383    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:28.099050 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:28.097192    2562 patch_prober.go:28] interesting pod/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 15 12:58:28.099050 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:28.097254    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 15 12:58:29.101618 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:29.101567    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:29.803392 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:29.802829    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:29.804882 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:29.804858    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:37.851832 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:37.851802    2562 generic.go:356] "Generic (PLEG): container finished" podID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerID="cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b" exitCode=0
Sep 15 12:58:37.852255 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:37.851879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" event={"ID":"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8","Type":"ContainerDied","Data":"cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b"}
Sep 15 12:58:37.852417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:37.852401    2562 scope.go:117] "RemoveContainer" containerID="cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b"
Sep 15 12:58:37.853631 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:37.853602    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:37.853720 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:37.853672    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:37.853720 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:37.853703    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.155210 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.155171    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"]
Sep 15 12:58:38.158494 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.158470    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.161441 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.161409    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1c35dd-openshift-default-dockercfg-m68cz\""
Sep 15 12:58:38.176309 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.176283    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"]
Sep 15 12:58:38.225238 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.225154    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-podinfo\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.225515 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.225492    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istiod-ca-cert\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.225642 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.225628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-token\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.225753 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.225739    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-envoy\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.225934 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.225920    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-certs\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.226089 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.226074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-socket\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.226213 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.226199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-credential-socket\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.226317 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.226303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-data\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.226421 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.226406    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vchsr\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-kube-api-access-vchsr\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.327926 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.327881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-certs\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.327942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-socket\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.327978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-credential-socket\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-data\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vchsr\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-kube-api-access-vchsr\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328140 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328138    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-podinfo\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istiod-ca-cert\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-token\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-envoy\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-certs\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328417 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-socket\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328712 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-credential-socket\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.328907 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.328831    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-data\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.329371 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.329351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istiod-ca-cert\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.330972 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.330945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-podinfo\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.331089 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.331048    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-envoy\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.336306 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.336280    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-token\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.336707 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.336691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vchsr\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-kube-api-access-vchsr\") pod \"e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") " pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.471631 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.471593    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:38.773524 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.773407    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"]
Sep 15 12:58:38.774723 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:58:38.774696    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda96fa1d9_5b27_4289_9913_bf4dbb7fe9d0.slice/crio-cd72a715ede33d5fb1996f7cd8337d220cd70c7264986ea2a8da42091f64f155 WatchSource:0}: Error finding container cd72a715ede33d5fb1996f7cd8337d220cd70c7264986ea2a8da42091f64f155: Status 404 returned error can't find the container with id cd72a715ede33d5fb1996f7cd8337d220cd70c7264986ea2a8da42091f64f155
Sep 15 12:58:38.777220 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.777187    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.777324 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.777252    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.777324 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.777278    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.796004 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.795974    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:58:38.859788 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.859759    2562 generic.go:356] "Generic (PLEG): container finished" podID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerID="0cc3e9026233ee8f798df7f436c675c8f66500e08892a0f43a871b65f67968c5" exitCode=0
Sep 15 12:58:38.860215 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.859825    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" event={"ID":"c527b9ee-4b9b-489e-b168-d985cbf47219","Type":"ContainerDied","Data":"0cc3e9026233ee8f798df7f436c675c8f66500e08892a0f43a871b65f67968c5"}
Sep 15 12:58:38.860277 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.860249    2562 scope.go:117] "RemoveContainer" containerID="0cc3e9026233ee8f798df7f436c675c8f66500e08892a0f43a871b65f67968c5"
Sep 15 12:58:38.861639 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.861604    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.861798 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.861694    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.861798 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.861751    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.862962 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.862943    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" event={"ID":"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8","Type":"ContainerStarted","Data":"288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038"}
Sep 15 12:58:38.864945 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.864921    2562 generic.go:356] "Generic (PLEG): container finished" podID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerID="354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27" exitCode=0
Sep 15 12:58:38.865076 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.864983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" event={"ID":"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c","Type":"ContainerDied","Data":"354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27"}
Sep 15 12:58:38.865380 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.865360    2562 scope.go:117] "RemoveContainer" containerID="354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27"
Sep 15 12:58:38.866513 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.866482    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" event={"ID":"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0","Type":"ContainerStarted","Data":"cd72a715ede33d5fb1996f7cd8337d220cd70c7264986ea2a8da42091f64f155"}
Sep 15 12:58:38.870502 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.870479    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.870648 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.870632    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:38.870758 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:38.870744    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:39.264013 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:39.263976    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:39.264339 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:39.264058    2562 patch_prober.go:28] interesting pod/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 12:58:39.264528 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:39.264498    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 12:58:39.879723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:39.879681    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" event={"ID":"c527b9ee-4b9b-489e-b168-d985cbf47219","Type":"ContainerStarted","Data":"94a30823d271f6fa1f870d462e86f645b4da338c757774323a8276869d62f195"}
Sep 15 12:58:39.881723 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:39.881700    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" event={"ID":"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c","Type":"ContainerStarted","Data":"8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68"}
Sep 15 12:58:39.883248 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:39.883225    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" event={"ID":"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0","Type":"ContainerStarted","Data":"7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68"}
Sep 15 12:58:39.929448 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:39.929369    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" podStartSLOduration=1.9293468219999999 podStartE2EDuration="1.929346822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:58:39.92753532 +0000 UTC m=+837.720237046" watchObservedRunningTime="2026-09-15 12:58:39.929346822 +0000 UTC m=+837.722048561"
Sep 15 12:58:40.264054 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.264001    2562 patch_prober.go:28] interesting pod/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 12:58:40.264262 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.264083    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 12:58:40.333303 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.333265    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:40.333724 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.333698    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 15 12:58:40.333927 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.333899    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 15 12:58:40.446496 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.446456    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:40.446859 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.446827    2562 patch_prober.go:28] interesting pod/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:58:40.447041 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.446996    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 15 12:58:40.471879 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.471845    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:40.488437 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.488410    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:40.895642 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.895609    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:40.900948 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:40.900921    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:58:41.263818 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.263773    2562 patch_prober.go:28] interesting pod/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 12:58:41.264060 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.263842    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 12:58:41.308504 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.308459    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"]
Sep 15 12:58:41.313054 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.313007    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:41.316453 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.316424    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 12:58:41.316675 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.316657    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 12:58:41.316889 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.316869    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 12:58:41.322093 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.322064    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"]
Sep 15 12:58:41.333489 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.333460    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 15 12:58:41.333717 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.333691    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 15 12:58:41.392763 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.392706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-08701e10-bkf2v\" (UID: \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:41.393093 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.393061    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f62b\" (UniqueName: \"kubernetes.io/projected/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-kube-api-access-7f62b\") pod \"maas-api-revoke-keys-e2e-ait-08701e10-bkf2v\" (UID: \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:41.483172 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.455527    2562 patch_prober.go:28] interesting pod/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:58:41.483172 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.455605    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 15 12:58:41.507055 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.501630    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-08701e10-bkf2v\" (UID: \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:41.507055 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.500636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-08701e10-bkf2v\" (UID: \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:41.507055 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.501740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f62b\" (UniqueName: \"kubernetes.io/projected/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-kube-api-access-7f62b\") pod \"maas-api-revoke-keys-e2e-ait-08701e10-bkf2v\" (UID: \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:41.515100 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.513704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f62b\" (UniqueName: \"kubernetes.io/projected/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-kube-api-access-7f62b\") pod \"maas-api-revoke-keys-e2e-ait-08701e10-bkf2v\" (UID: \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:41.634143 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:41.634094    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:42.246109 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.245183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"]
Sep 15 12:58:42.270712 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.266544    2562 patch_prober.go:28] interesting pod/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 12:58:42.270712 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.266619    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 12:58:42.279440 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.279397    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:58:42.338055 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.335453    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 15 12:58:42.338055 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.335519    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 15 12:58:42.447259 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.447216    2562 patch_prober.go:28] interesting pod/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:58:42.447483 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.447289    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 15 12:58:42.921575 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:42.921499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v" event={"ID":"83fc41f7-c8f2-4b16-afdc-f4a701c05c16","Type":"ContainerStarted","Data":"60ee58c4322a40a630f31bc9a4011bfe227b9bb8a7453bb352e566fa7861fdda"}
Sep 15 12:58:43.265924 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:43.265805    2562 patch_prober.go:28] interesting pod/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 12:58:43.267011 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:43.266978    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 12:58:43.336562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:43.336520    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 15 12:58:43.336859 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:43.336829    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 15 12:58:43.448388 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:43.448345    2562 patch_prober.go:28] interesting pod/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:58:43.448686 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:43.448641    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 15 12:58:44.301233 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:44.301192    2562 patch_prober.go:28] interesting pod/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 12:58:44.301887 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:44.301846    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 12:58:44.337054 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:44.334489    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 15 12:58:44.337054 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:44.334563    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 15 12:58:44.447235 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:44.446695    2562 patch_prober.go:28] interesting pod/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:58:44.447235 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:44.446769    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 15 12:58:45.265057 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:45.263430    2562 patch_prober.go:28] interesting pod/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 12:58:45.265057 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:45.263503    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 12:58:45.340052 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:45.333337    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 15 12:58:45.340052 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:45.333401    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 15 12:58:45.446494 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:45.446427    2562 patch_prober.go:28] interesting pod/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:58:45.446702 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:45.446515    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 15 12:58:45.667286 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:45.667136    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:46.313853 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:46.312331    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:46.322574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:46.322490    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 12:58:46.337582 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:46.337528    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 15 12:58:46.338314 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:46.337862    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 15 12:58:46.446557 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:46.446524    2562 patch_prober.go:28] interesting pod/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:58:46.447182 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:46.447125    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 15 12:58:46.669192 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:46.669083    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:46.670548 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:46.670526    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:47.356101 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:47.341273    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 15 12:58:47.356101 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:47.341331    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 15 12:58:47.447997 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:47.447536    2562 patch_prober.go:28] interesting pod/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 15 12:58:47.447997 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:47.447601    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 15 12:58:48.342780 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:48.341314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:48.354794 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:48.351463    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 12:58:48.456061 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:48.453266    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:48.456061 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:48.455903    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 12:58:50.020348 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:50.020249    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v" event={"ID":"83fc41f7-c8f2-4b16-afdc-f4a701c05c16","Type":"ContainerStarted","Data":"562df64bfd1af5efc1491ed132800fa2bc7a5b1d8b1efbb1f7373b08f7c01ab0"}
Sep 15 12:58:51.031061 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:51.030117    2562 generic.go:356] "Generic (PLEG): container finished" podID="83fc41f7-c8f2-4b16-afdc-f4a701c05c16" containerID="562df64bfd1af5efc1491ed132800fa2bc7a5b1d8b1efbb1f7373b08f7c01ab0" exitCode=0
Sep 15 12:58:51.031061 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:51.030200    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v" event={"ID":"83fc41f7-c8f2-4b16-afdc-f4a701c05c16","Type":"ContainerDied","Data":"562df64bfd1af5efc1491ed132800fa2bc7a5b1d8b1efbb1f7373b08f7c01ab0"}
Sep 15 12:58:52.225806 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:52.225769    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:52.331674 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:52.331583    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-maas-api-service-ca\") pod \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\" (UID: \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\") "
Sep 15 12:58:52.334115 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:52.331739    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f62b\" (UniqueName: \"kubernetes.io/projected/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-kube-api-access-7f62b\") pod \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\" (UID: \"83fc41f7-c8f2-4b16-afdc-f4a701c05c16\") "
Sep 15 12:58:52.334115 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:52.331966    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "83fc41f7-c8f2-4b16-afdc-f4a701c05c16" (UID: "83fc41f7-c8f2-4b16-afdc-f4a701c05c16"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:58:52.334115 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:52.332115    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:58:52.334115 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:52.333995    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-kube-api-access-7f62b" (OuterVolumeSpecName: "kube-api-access-7f62b") pod "83fc41f7-c8f2-4b16-afdc-f4a701c05c16" (UID: "83fc41f7-c8f2-4b16-afdc-f4a701c05c16"). InnerVolumeSpecName "kube-api-access-7f62b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:58:52.433452 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:52.433413    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f62b\" (UniqueName: \"kubernetes.io/projected/83fc41f7-c8f2-4b16-afdc-f4a701c05c16-kube-api-access-7f62b\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:58:53.044595 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.044514    2562 generic.go:356] "Generic (PLEG): container finished" podID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerID="2b0bc502963b7c5293facba8178b522f79fc83b4cf3e5dd01f9cfff930ae90d3" exitCode=0
Sep 15 12:58:53.044739 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.044591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" event={"ID":"cf9f1d07-f711-4d0c-9961-a2823c61b721","Type":"ContainerDied","Data":"2b0bc502963b7c5293facba8178b522f79fc83b4cf3e5dd01f9cfff930ae90d3"}
Sep 15 12:58:53.045167 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.045142    2562 scope.go:117] "RemoveContainer" containerID="2b0bc502963b7c5293facba8178b522f79fc83b4cf3e5dd01f9cfff930ae90d3"
Sep 15 12:58:53.046231 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.046206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v" event={"ID":"83fc41f7-c8f2-4b16-afdc-f4a701c05c16","Type":"ContainerDied","Data":"60ee58c4322a40a630f31bc9a4011bfe227b9bb8a7453bb352e566fa7861fdda"}
Sep 15 12:58:53.046354 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.046252    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60ee58c4322a40a630f31bc9a4011bfe227b9bb8a7453bb352e566fa7861fdda"
Sep 15 12:58:53.046354 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.046282    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"
Sep 15 12:58:53.046667 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.046631    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:53.046752 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.046729    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:53.046806 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:53.046779    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:54.054668 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:54.054626    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" event={"ID":"cf9f1d07-f711-4d0c-9961-a2823c61b721","Type":"ContainerStarted","Data":"2364ec0eb993aefd929a3de9a6620d9f1191bf63b8be103f47015bc333991124"}
Sep 15 12:58:54.966113 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:54.966073    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:54.967244 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:54.967212    2562 patch_prober.go:28] interesting pod/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:58:54.967380 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:54.967271    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 15 12:58:55.063215 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:55.063183    2562 generic.go:356] "Generic (PLEG): container finished" podID="c77919fd-12dc-4604-9e31-f62777722f3b" containerID="5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350" exitCode=0
Sep 15 12:58:55.063593 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:55.063259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" event={"ID":"c77919fd-12dc-4604-9e31-f62777722f3b","Type":"ContainerDied","Data":"5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350"}
Sep 15 12:58:55.063813 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:55.063799    2562 scope.go:117] "RemoveContainer" containerID="5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350"
Sep 15 12:58:55.064950 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:55.064921    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:55.065017 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:55.064992    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:55.065083 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:55.065023    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:58:55.966562 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:55.966525    2562 patch_prober.go:28] interesting pod/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:58:55.966842 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:55.966812    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 15 12:58:56.070693 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:56.070659    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" event={"ID":"c77919fd-12dc-4604-9e31-f62777722f3b","Type":"ContainerStarted","Data":"3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d"}
Sep 15 12:58:56.090496 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:56.090471    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:56.091392 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:56.091372    2562 patch_prober.go:28] interesting pod/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 15 12:58:56.091477 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:56.091423    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 15 12:58:56.744393 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:56.744356    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:56.966828 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:56.966409    2562 patch_prober.go:28] interesting pod/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:58:56.966828 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:56.966474    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 15 12:58:57.091211 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:57.091137    2562 patch_prober.go:28] interesting pod/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 15 12:58:57.091211 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:57.091198    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 15 12:58:57.965731 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:57.965697    2562 patch_prober.go:28] interesting pod/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:58:57.966077 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:57.966026    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 15 12:58:58.090987 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:58.090957    2562 patch_prober.go:28] interesting pod/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 15 12:58:58.091203 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:58.091163    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 15 12:58:58.965673 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:58.965635    2562 patch_prober.go:28] interesting pod/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 15 12:58:58.966302 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:58.966269    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 15 12:58:59.096671 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:59.096635    2562 patch_prober.go:28] interesting pod/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 15 12:58:59.096954 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:59.096927    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 15 12:58:59.791531 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:59.791493    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:58:59.976162 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:59.976128    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:58:59.977411 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:58:59.977387    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:59:00.091452 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:00.091362    2562 patch_prober.go:28] interesting pod/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 15 12:59:00.091452 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:00.091437    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 15 12:59:01.090818 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:01.090779    2562 patch_prober.go:28] interesting pod/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 15 12:59:01.091299 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:01.090844    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 15 12:59:02.095219 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:02.095182    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:59:02.103080 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:02.103052    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 12:59:02.550055 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:02.549475    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"]
Sep 15 12:59:02.551711 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:02.551683    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-08701e10-bkf2v"]
Sep 15 12:59:02.700369 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:02.700333    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"]
Sep 15 12:59:02.700826 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:02.700798    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy" containerID="cri-o://2364ec0eb993aefd929a3de9a6620d9f1191bf63b8be103f47015bc333991124" gracePeriod=30
Sep 15 12:59:02.766985 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:02.766950    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83fc41f7-c8f2-4b16-afdc-f4a701c05c16" path="/var/lib/kubelet/pods/83fc41f7-c8f2-4b16-afdc-f4a701c05c16/volumes"
Sep 15 12:59:03.616781 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.616751    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"]
Sep 15 12:59:03.617250 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.617237    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83fc41f7-c8f2-4b16-afdc-f4a701c05c16" containerName="revoke-keys"
Sep 15 12:59:03.617250 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.617251    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="83fc41f7-c8f2-4b16-afdc-f4a701c05c16" containerName="revoke-keys"
Sep 15 12:59:03.617322 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.617316    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="83fc41f7-c8f2-4b16-afdc-f4a701c05c16" containerName="revoke-keys"
Sep 15 12:59:03.629377 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.629348    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.649083 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.649055    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f78f068d-gw-openshift-default-dockercfg-6xfq7\""
Sep 15 12:59:03.657440 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.657305    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"]
Sep 15 12:59:03.781048 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.780987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-credential-socket\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.781248 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.781105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-podinfo\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.781248 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.781141    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-data\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.781248 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.781179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-envoy\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.781248 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.781203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe2218d-e55d-4e55-8612-3226a65d788a-istiod-ca-cert\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.781556 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.781296    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-certs\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.781556 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.781341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-token\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.781556 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.781419    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv9hm\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-kube-api-access-wv9hm\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.781556 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.781469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-socket\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882495 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv9hm\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-kube-api-access-wv9hm\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882495 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-socket\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882495 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-credential-socket\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882804 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-podinfo\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882804 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-data\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882804 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882551    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-envoy\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882804 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe2218d-e55d-4e55-8612-3226a65d788a-istiod-ca-cert\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882804 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-certs\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.882804 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882698    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-token\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.883156 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-credential-socket\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.883156 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.882987    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-data\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.883156 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.883057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-socket\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.883156 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.883139    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-certs\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.883533 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.883512    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe2218d-e55d-4e55-8612-3226a65d788a-istiod-ca-cert\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.884871 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.884845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-envoy\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.885072 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.885057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-podinfo\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.895779 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.895749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-token\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.896121 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.896085    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv9hm\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-kube-api-access-wv9hm\") pod \"e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") " pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:03.950945 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:03.950909    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:04.242243 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:04.242217    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"]
Sep 15 12:59:04.244091 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:59:04.244063    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe2218d_e55d_4e55_8612_3226a65d788a.slice/crio-d5986f11e9a51956859c73a07cb699df8d3e8637dbdd6a920cf4fe9b74350e45 WatchSource:0}: Error finding container d5986f11e9a51956859c73a07cb699df8d3e8637dbdd6a920cf4fe9b74350e45: Status 404 returned error can't find the container with id d5986f11e9a51956859c73a07cb699df8d3e8637dbdd6a920cf4fe9b74350e45
Sep 15 12:59:04.246479 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:04.246448    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:04.246560 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:04.246514    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:04.246600 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:04.246555    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:05.132976 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:05.132935    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" event={"ID":"bbe2218d-e55d-4e55-8612-3226a65d788a","Type":"ContainerStarted","Data":"af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6"}
Sep 15 12:59:05.133405 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:05.133002    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" event={"ID":"bbe2218d-e55d-4e55-8612-3226a65d788a","Type":"ContainerStarted","Data":"d5986f11e9a51956859c73a07cb699df8d3e8637dbdd6a920cf4fe9b74350e45"}
Sep 15 12:59:05.162961 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:05.162900    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" podStartSLOduration=2.162879355 podStartE2EDuration="2.162879355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:59:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:59:05.161026805 +0000 UTC m=+862.953728548" watchObservedRunningTime="2026-09-15 12:59:05.162879355 +0000 UTC m=+862.955581076"
Sep 15 12:59:05.951269 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:05.951230    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:05.956414 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:05.956386    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:06.139574 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:06.139543    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:06.140498 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:06.140476    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:08.153984 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.153954    2562 generic.go:356] "Generic (PLEG): container finished" podID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerID="2364ec0eb993aefd929a3de9a6620d9f1191bf63b8be103f47015bc333991124" exitCode=0
Sep 15 12:59:08.154527 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.154076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" event={"ID":"cf9f1d07-f711-4d0c-9961-a2823c61b721","Type":"ContainerDied","Data":"2364ec0eb993aefd929a3de9a6620d9f1191bf63b8be103f47015bc333991124"}
Sep 15 12:59:08.154527 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.154158    2562 scope.go:117] "RemoveContainer" containerID="2b0bc502963b7c5293facba8178b522f79fc83b4cf3e5dd01f9cfff930ae90d3"
Sep 15 12:59:08.156385 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.156355    2562 generic.go:356] "Generic (PLEG): container finished" podID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerID="7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68" exitCode=0
Sep 15 12:59:08.156673 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.156646    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" event={"ID":"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0","Type":"ContainerDied","Data":"7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68"}
Sep 15 12:59:08.157220 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.157198    2562 scope.go:117] "RemoveContainer" containerID="7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68"
Sep 15 12:59:08.159463 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.159426    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:08.159535 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.159519    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:08.159585 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.159564    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:08.205974 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.205950    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:59:08.336333 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336303    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-token\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336363    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-data\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336381    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-podinfo\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-socket\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336471    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-certs\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336505    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-envoy\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336786 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336534    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf9f1d07-f711-4d0c-9961-a2823c61b721-istiod-ca-cert\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336786 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336585    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-credential-socket\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336786 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336602    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lggsf\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-kube-api-access-lggsf\") pod \"cf9f1d07-f711-4d0c-9961-a2823c61b721\" (UID: \"cf9f1d07-f711-4d0c-9961-a2823c61b721\") "
Sep 15 12:59:08.336786 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336624    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:08.337004 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.336788    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:08.337082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.337007    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:08.337553 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.337165    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:08.337553 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.337192    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:08.337553 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.337208    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:08.337553 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.337504    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:08.337913 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.337885    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf9f1d07-f711-4d0c-9961-a2823c61b721-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:59:08.339244 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.339198    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:08.340142 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.340098    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:59:08.340142 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.340125    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-kube-api-access-lggsf" (OuterVolumeSpecName: "kube-api-access-lggsf") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "kube-api-access-lggsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:59:08.340256 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.340207    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf9f1d07-f711-4d0c-9961-a2823c61b721" (UID: "cf9f1d07-f711-4d0c-9961-a2823c61b721"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:59:08.438424 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.438333    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:08.438424 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.438362    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:08.438424 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.438372    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf9f1d07-f711-4d0c-9961-a2823c61b721-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:08.438424 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.438380    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lggsf\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-kube-api-access-lggsf\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:08.438424 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.438390    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:08.438424 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:08.438398    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf9f1d07-f711-4d0c-9961-a2823c61b721-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:09.170893 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.170790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" event={"ID":"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0","Type":"ContainerStarted","Data":"df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de"}
Sep 15 12:59:09.172453 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.172396    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v" event={"ID":"cf9f1d07-f711-4d0c-9961-a2823c61b721","Type":"ContainerDied","Data":"b7e40021f13dcce4129febd417027e44bda17ed5ef511a0e8357335e75aef198"}
Sep 15 12:59:09.172582 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.172451    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"
Sep 15 12:59:09.172582 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.172479    2562 scope.go:117] "RemoveContainer" containerID="2364ec0eb993aefd929a3de9a6620d9f1191bf63b8be103f47015bc333991124"
Sep 15 12:59:09.206234 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.206205    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"]
Sep 15 12:59:09.210726 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.210695    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-08701e10-openshift-default-6597cdf99b-8jx4v"]
Sep 15 12:59:09.472120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.472083    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:59:09.472923 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.472899    2562 patch_prober.go:28] interesting pod/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 15 12:59:09.473052 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:09.472949    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 15 12:59:10.472829 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:10.472791    2562 patch_prober.go:28] interesting pod/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 15 12:59:10.473247 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:10.472850    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 15 12:59:10.756336 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:10.756258    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" path="/var/lib/kubelet/pods/cf9f1d07-f711-4d0c-9961-a2823c61b721/volumes"
Sep 15 12:59:10.895719 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:10.895686    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:59:10.905370 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:10.905333    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:59:11.472743 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:11.472708    2562 patch_prober.go:28] interesting pod/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 15 12:59:11.472913 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:11.472759    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 15 12:59:12.473014 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:12.472944    2562 patch_prober.go:28] interesting pod/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 15 12:59:12.473451 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:12.473009    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 15 12:59:13.472768 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:13.472735    2562 patch_prober.go:28] interesting pod/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 15 12:59:13.472961 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:13.472785    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 15 12:59:14.472017 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:14.471977    2562 patch_prober.go:28] interesting pod/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 15 12:59:14.472430 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:14.472079    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 15 12:59:15.476728 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:15.476694    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:59:15.477743 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:15.477720    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 12:59:30.686589 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:30.686552    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 12:59:31.638353 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.638321    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"]
Sep 15 12:59:31.638775 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.638762    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy"
Sep 15 12:59:31.638775 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.638776    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy"
Sep 15 12:59:31.638900 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.638864    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy"
Sep 15 12:59:31.638900 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.638872    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy"
Sep 15 12:59:31.640931 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.640913    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:31.646670 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.646652    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 12:59:31.647007 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.646991    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 12:59:31.647717 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.647698    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 12:59:31.656152 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.656128    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"]
Sep 15 12:59:31.810278 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.810238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn\" (UID: \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:31.810278 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.810279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78rgq\" (UniqueName: \"kubernetes.io/projected/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-kube-api-access-78rgq\") pod \"maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn\" (UID: \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:31.910793 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.910767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn\" (UID: \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:31.911830 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.910798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78rgq\" (UniqueName: \"kubernetes.io/projected/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-kube-api-access-78rgq\") pod \"maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn\" (UID: \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:31.911830 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.911392    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn\" (UID: \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:31.920499 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.920465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78rgq\" (UniqueName: \"kubernetes.io/projected/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-kube-api-access-78rgq\") pod \"maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn\" (UID: \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:31.981832 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:31.981806    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:32.260435 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:32.260407    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"]
Sep 15 12:59:32.262222 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:59:32.262195    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a28fd6_551a_4be2_9ee0_799cbbf506d4.slice/crio-6e13598c0731f89e3ce923a03d1c1663d3f020d3901fc60f0b68afffe1a18e27 WatchSource:0}: Error finding container 6e13598c0731f89e3ce923a03d1c1663d3f020d3901fc60f0b68afffe1a18e27: Status 404 returned error can't find the container with id 6e13598c0731f89e3ce923a03d1c1663d3f020d3901fc60f0b68afffe1a18e27
Sep 15 12:59:32.303799 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:32.303771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn" event={"ID":"a8a28fd6-551a-4be2-9ee0-799cbbf506d4","Type":"ContainerStarted","Data":"6e13598c0731f89e3ce923a03d1c1663d3f020d3901fc60f0b68afffe1a18e27"}
Sep 15 12:59:32.305478 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:32.305456    2562 generic.go:356] "Generic (PLEG): container finished" podID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerID="af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6" exitCode=0
Sep 15 12:59:32.305602 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:32.305513    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" event={"ID":"bbe2218d-e55d-4e55-8612-3226a65d788a","Type":"ContainerDied","Data":"af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6"}
Sep 15 12:59:32.305947 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:32.305930    2562 scope.go:117] "RemoveContainer" containerID="af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6"
Sep 15 12:59:32.307045 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:32.307004    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:32.307125 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:32.307088    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:32.307164 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:32.307126    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:33.312874 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:33.312786    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" event={"ID":"bbe2218d-e55d-4e55-8612-3226a65d788a","Type":"ContainerStarted","Data":"953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f"}
Sep 15 12:59:33.314316 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:33.314290    2562 generic.go:356] "Generic (PLEG): container finished" podID="a8a28fd6-551a-4be2-9ee0-799cbbf506d4" containerID="c30a1857ee964685473e8586c2a373ed063300a041e00cc8eb67fce26d35bf7a" exitCode=0
Sep 15 12:59:33.314433 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:33.314369    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn" event={"ID":"a8a28fd6-551a-4be2-9ee0-799cbbf506d4","Type":"ContainerDied","Data":"c30a1857ee964685473e8586c2a373ed063300a041e00cc8eb67fce26d35bf7a"}
Sep 15 12:59:33.952145 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:33.952091    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:33.953401 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:33.953365    2562 patch_prober.go:28] interesting pod/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:59:33.953569 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:33.953435    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 15 12:59:34.506914 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.506887    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:34.651566 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.651486    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-maas-api-service-ca\") pod \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\" (UID: \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\") "
Sep 15 12:59:34.651711 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.651602    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78rgq\" (UniqueName: \"kubernetes.io/projected/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-kube-api-access-78rgq\") pod \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\" (UID: \"a8a28fd6-551a-4be2-9ee0-799cbbf506d4\") "
Sep 15 12:59:34.651918 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.651891    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a8a28fd6-551a-4be2-9ee0-799cbbf506d4" (UID: "a8a28fd6-551a-4be2-9ee0-799cbbf506d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:59:34.652194 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.652171    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:34.655120 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.654348    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-kube-api-access-78rgq" (OuterVolumeSpecName: "kube-api-access-78rgq") pod "a8a28fd6-551a-4be2-9ee0-799cbbf506d4" (UID: "a8a28fd6-551a-4be2-9ee0-799cbbf506d4"). InnerVolumeSpecName "kube-api-access-78rgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:59:34.753119 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.753088    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78rgq\" (UniqueName: \"kubernetes.io/projected/a8a28fd6-551a-4be2-9ee0-799cbbf506d4-kube-api-access-78rgq\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:34.951881 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.951845    2562 patch_prober.go:28] interesting pod/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:59:34.952073 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:34.951898    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 15 12:59:35.330274 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:35.330170    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn" event={"ID":"a8a28fd6-551a-4be2-9ee0-799cbbf506d4","Type":"ContainerDied","Data":"6e13598c0731f89e3ce923a03d1c1663d3f020d3901fc60f0b68afffe1a18e27"}
Sep 15 12:59:35.330274 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:35.330259    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e13598c0731f89e3ce923a03d1c1663d3f020d3901fc60f0b68afffe1a18e27"
Sep 15 12:59:35.330576 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:35.330350    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"
Sep 15 12:59:35.952256 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:35.952219    2562 patch_prober.go:28] interesting pod/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:59:35.952630 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:35.952278    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 15 12:59:36.140220 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:36.140181    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:36.952008 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:36.951973    2562 patch_prober.go:28] interesting pod/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:59:36.952208 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:36.952056    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 15 12:59:37.951900 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:37.951861    2562 patch_prober.go:28] interesting pod/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 15 12:59:37.952315 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:37.951923    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 15 12:59:38.626226 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:38.626188    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"]
Sep 15 12:59:38.630341 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:38.630315    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f78f068d-t2wnn"]
Sep 15 12:59:38.764212 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:38.764021    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8a28fd6-551a-4be2-9ee0-799cbbf506d4" path="/var/lib/kubelet/pods/a8a28fd6-551a-4be2-9ee0-799cbbf506d4/volumes"
Sep 15 12:59:38.764928 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:38.764836    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"]
Sep 15 12:59:38.765716 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:38.765626    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy" containerID="cri-o://953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f" gracePeriod=30
Sep 15 12:59:39.483480 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.483377    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"]
Sep 15 12:59:39.484082 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.484063    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8a28fd6-551a-4be2-9ee0-799cbbf506d4" containerName="revoke-keys"
Sep 15 12:59:39.484145 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.484084    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8a28fd6-551a-4be2-9ee0-799cbbf506d4" containerName="revoke-keys"
Sep 15 12:59:39.484145 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.484107    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy"
Sep 15 12:59:39.484145 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.484115    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf9f1d07-f711-4d0c-9961-a2823c61b721" containerName="istio-proxy"
Sep 15 12:59:39.484252 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.484241    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8a28fd6-551a-4be2-9ee0-799cbbf506d4" containerName="revoke-keys"
Sep 15 12:59:39.486360 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.486340    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.489232 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.489212    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dc842c6b-openshift-default-dockercfg-cmngb\""
Sep 15 12:59:39.500251 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.500231    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"]
Sep 15 12:59:39.503293 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xmmb\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-kube-api-access-2xmmb\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.503392 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-socket\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.503392 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503317    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-istio-token\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.503392 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78225ab2-03b3-4575-8231-c81f59e5918e-istio-podinfo\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.503510 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-data\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.503510 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78225ab2-03b3-4575-8231-c81f59e5918e-istiod-ca-cert\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.503588 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503545    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-certs\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.503588 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-credential-socket\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.503695 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.503647    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-envoy\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.604931 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.604890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78225ab2-03b3-4575-8231-c81f59e5918e-istio-podinfo\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.605145 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.605021    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-data\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.605145 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.605070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78225ab2-03b3-4575-8231-c81f59e5918e-istiod-ca-cert\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.605145 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.605138    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-certs\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.605319 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.605193    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-credential-socket\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.605319 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.605235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-envoy\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.605319 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.605275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xmmb\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-kube-api-access-2xmmb\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.605319 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.605301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-socket\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.605512 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.605323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-istio-token\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.606481 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.606448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-credential-socket\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.606773 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.606741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-data\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.607366 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.607337    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78225ab2-03b3-4575-8231-c81f59e5918e-istiod-ca-cert\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.607587 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.607560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-certs\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.608532 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.608508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-socket\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.619758 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.619724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-envoy\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.621336 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.620234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78225ab2-03b3-4575-8231-c81f59e5918e-istio-podinfo\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.622162 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.622137    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xmmb\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-kube-api-access-2xmmb\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.623818 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.623794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-istio-token\") pod \"e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") " pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:39.797148 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:39.797064    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:40.079395 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:40.079370    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"]
Sep 15 12:59:40.080964 ip-10-0-132-169 kubenswrapper[2562]: W0915 12:59:40.080939    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78225ab2_03b3_4575_8231_c81f59e5918e.slice/crio-f6d089e7ad7b6968054f32c9a09674ee17fd79c2646a55cd4659c9c6b366f601 WatchSource:0}: Error finding container f6d089e7ad7b6968054f32c9a09674ee17fd79c2646a55cd4659c9c6b366f601: Status 404 returned error can't find the container with id f6d089e7ad7b6968054f32c9a09674ee17fd79c2646a55cd4659c9c6b366f601
Sep 15 12:59:40.083240 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:40.083210    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:40.083306 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:40.083271    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:40.083347 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:40.083317    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 12:59:40.384075 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:40.383968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" event={"ID":"78225ab2-03b3-4575-8231-c81f59e5918e","Type":"ContainerStarted","Data":"7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3"}
Sep 15 12:59:40.384075 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:40.384055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" event={"ID":"78225ab2-03b3-4575-8231-c81f59e5918e","Type":"ContainerStarted","Data":"f6d089e7ad7b6968054f32c9a09674ee17fd79c2646a55cd4659c9c6b366f601"}
Sep 15 12:59:40.419109 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:40.419050    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" podStartSLOduration=1.419015443 podStartE2EDuration="1.419015443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:59:40.414086473 +0000 UTC m=+898.206788209" watchObservedRunningTime="2026-09-15 12:59:40.419015443 +0000 UTC m=+898.211717178"
Sep 15 12:59:40.797474 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:40.797443    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:41.803975 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:41.803942    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:42.395460 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:42.395431    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:42.396475 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:42.396453    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 12:59:42.706291 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:42.706253    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:59:42.707024 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:42.707000    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 12:59:42.710268 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:42.710249    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:59:42.711255 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:42.711235    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 12:59:44.237341 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.237315    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:44.366741 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.366652    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-token\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.366891 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.366759    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-credential-socket\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.366891 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.366795    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-certs\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.366891 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.366835    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-data\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.366891 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.366861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe2218d-e55d-4e55-8612-3226a65d788a-istiod-ca-cert\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.367133 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.366897    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-socket\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.367133 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.366921    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-envoy\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.367133 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.366949    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-podinfo\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.367133 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367023    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv9hm\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-kube-api-access-wv9hm\") pod \"bbe2218d-e55d-4e55-8612-3226a65d788a\" (UID: \"bbe2218d-e55d-4e55-8612-3226a65d788a\") "
Sep 15 12:59:44.367133 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367068    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:44.367133 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367078    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:44.367133 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367117    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-data" (OuterVolumeSpecName: "istio-data") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:44.367495 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367382    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe2218d-e55d-4e55-8612-3226a65d788a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:59:44.367554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367477    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:44.367554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367499    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.367554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367520    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.367554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367533    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe2218d-e55d-4e55-8612-3226a65d788a-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.367554 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.367546    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.369009 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.368985    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-token" (OuterVolumeSpecName: "istio-token") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:59:44.369301 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.369278    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:59:44.369519 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.369503    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:59:44.369566 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.369536    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-kube-api-access-wv9hm" (OuterVolumeSpecName: "kube-api-access-wv9hm") pod "bbe2218d-e55d-4e55-8612-3226a65d788a" (UID: "bbe2218d-e55d-4e55-8612-3226a65d788a"). InnerVolumeSpecName "kube-api-access-wv9hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:59:44.412225 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.412194    2562 generic.go:356] "Generic (PLEG): container finished" podID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerID="953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f" exitCode=0
Sep 15 12:59:44.412359 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.412244    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"
Sep 15 12:59:44.412359 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.412277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" event={"ID":"bbe2218d-e55d-4e55-8612-3226a65d788a","Type":"ContainerDied","Data":"953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f"}
Sep 15 12:59:44.412430 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.412366    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8" event={"ID":"bbe2218d-e55d-4e55-8612-3226a65d788a","Type":"ContainerDied","Data":"d5986f11e9a51956859c73a07cb699df8d3e8637dbdd6a920cf4fe9b74350e45"}
Sep 15 12:59:44.412430 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.412396    2562 scope.go:117] "RemoveContainer" containerID="953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f"
Sep 15 12:59:44.426693 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:59:44.426659    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe2218d_e55d_4e55_8612_3226a65d788a.slice/crio-d5986f11e9a51956859c73a07cb699df8d3e8637dbdd6a920cf4fe9b74350e45\": RecentStats: unable to find data in memory cache]"
Sep 15 12:59:44.432969 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.432946    2562 scope.go:117] "RemoveContainer" containerID="af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6"
Sep 15 12:59:44.441641 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.441618    2562 scope.go:117] "RemoveContainer" containerID="953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f"
Sep 15 12:59:44.442077 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:59:44.442048    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f\": container with ID starting with 953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f not found: ID does not exist" containerID="953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f"
Sep 15 12:59:44.442200 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.442129    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f"} err="failed to get container status \"953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f\": rpc error: code = NotFound desc = could not find container \"953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f\": container with ID starting with 953db647d2db31ae7565cf2bce69fab9e293d21b370751010dd1a1f84824323f not found: ID does not exist"
Sep 15 12:59:44.442200 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.442156    2562 scope.go:117] "RemoveContainer" containerID="af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6"
Sep 15 12:59:44.442677 ip-10-0-132-169 kubenswrapper[2562]: E0915 12:59:44.442651    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6\": container with ID starting with af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6 not found: ID does not exist" containerID="af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6"
Sep 15 12:59:44.442760 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.442684    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6"} err="failed to get container status \"af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6\": rpc error: code = NotFound desc = could not find container \"af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6\": container with ID starting with af73a546dbe0e4303f04970732715031303b33e2250717d0ac37cdfcb8bf78c6 not found: ID does not exist"
Sep 15 12:59:44.445206 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.445184    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"]
Sep 15 12:59:44.450726 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.450705    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f78f068d-gw-openshift-default-66488f974d-kx2d8"]
Sep 15 12:59:44.469026 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.468994    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.469026 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.469022    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.469258 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.469064    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.469258 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.469078    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv9hm\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-kube-api-access-wv9hm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.469258 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.469091    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe2218d-e55d-4e55-8612-3226a65d788a-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 12:59:44.756967 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:44.756926    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" path="/var/lib/kubelet/pods/bbe2218d-e55d-4e55-8612-3226a65d788a/volumes"
Sep 15 12:59:54.227912 ip-10-0-132-169 kubenswrapper[2562]: I0915 12:59:54.227872    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:00:00.143728 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.143693    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"]
Sep 15 13:00:00.144328 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.144308    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy"
Sep 15 13:00:00.144401 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.144334    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy"
Sep 15 13:00:00.144538 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.144520    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy"
Sep 15 13:00:00.144607 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.144541    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy"
Sep 15 13:00:00.157422 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.157359    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"]
Sep 15 13:00:00.157732 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.157710    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"
Sep 15 13:00:00.158046 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.158012    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy"
Sep 15 13:00:00.158200 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.158188    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe2218d-e55d-4e55-8612-3226a65d788a" containerName="istio-proxy"
Sep 15 13:00:00.161149 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.161124    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:00:00.161460 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.161141    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:00:00.161737 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.161144    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"]
Sep 15 13:00:00.162163 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.161215    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"
Sep 15 13:00:00.162264 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.162190    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:00:00.166832 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.166810    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"]
Sep 15 13:00:00.167437 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.167416    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"]
Sep 15 13:00:00.167644 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.167622    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"]
Sep 15 13:00:00.167749 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.167057    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"
Sep 15 13:00:00.176411 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.176356    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"]
Sep 15 13:00:00.179820 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.179799    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"
Sep 15 13:00:00.186527 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.186494    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"]
Sep 15 13:00:00.242434 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.242402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2dwx\" (UniqueName: \"kubernetes.io/projected/16abf046-395a-4e20-9bd3-74be0b9732c0-kube-api-access-d2dwx\") pod \"maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6\" (UID: \"16abf046-395a-4e20-9bd3-74be0b9732c0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"
Sep 15 13:00:00.242597 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.242518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2fjc\" (UniqueName: \"kubernetes.io/projected/35b2317b-dc8a-4d60-9ec6-9621e2102196-kube-api-access-l2fjc\") pod \"maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv\" (UID: \"35b2317b-dc8a-4d60-9ec6-9621e2102196\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"
Sep 15 13:00:00.242597 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.242556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s75xm\" (UniqueName: \"kubernetes.io/projected/a744131f-9791-45b8-9815-537ab9b389ae-kube-api-access-s75xm\") pod \"maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc\" (UID: \"a744131f-9791-45b8-9815-537ab9b389ae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"
Sep 15 13:00:00.242597 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.242590    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4d9l\" (UniqueName: \"kubernetes.io/projected/6d332f41-9e18-445e-8dd9-2b660922fe62-kube-api-access-r4d9l\") pod \"maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd\" (UID: \"6d332f41-9e18-445e-8dd9-2b660922fe62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"
Sep 15 13:00:00.343207 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.343171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2dwx\" (UniqueName: \"kubernetes.io/projected/16abf046-395a-4e20-9bd3-74be0b9732c0-kube-api-access-d2dwx\") pod \"maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6\" (UID: \"16abf046-395a-4e20-9bd3-74be0b9732c0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"
Sep 15 13:00:00.343394 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.343274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2fjc\" (UniqueName: \"kubernetes.io/projected/35b2317b-dc8a-4d60-9ec6-9621e2102196-kube-api-access-l2fjc\") pod \"maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv\" (UID: \"35b2317b-dc8a-4d60-9ec6-9621e2102196\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"
Sep 15 13:00:00.343394 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.343307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s75xm\" (UniqueName: \"kubernetes.io/projected/a744131f-9791-45b8-9815-537ab9b389ae-kube-api-access-s75xm\") pod \"maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc\" (UID: \"a744131f-9791-45b8-9815-537ab9b389ae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"
Sep 15 13:00:00.343394 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.343343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4d9l\" (UniqueName: \"kubernetes.io/projected/6d332f41-9e18-445e-8dd9-2b660922fe62-kube-api-access-r4d9l\") pod \"maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd\" (UID: \"6d332f41-9e18-445e-8dd9-2b660922fe62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"
Sep 15 13:00:00.352187 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.352147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2fjc\" (UniqueName: \"kubernetes.io/projected/35b2317b-dc8a-4d60-9ec6-9621e2102196-kube-api-access-l2fjc\") pod \"maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv\" (UID: \"35b2317b-dc8a-4d60-9ec6-9621e2102196\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"
Sep 15 13:00:00.352439 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.352418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4d9l\" (UniqueName: \"kubernetes.io/projected/6d332f41-9e18-445e-8dd9-2b660922fe62-kube-api-access-r4d9l\") pod \"maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd\" (UID: \"6d332f41-9e18-445e-8dd9-2b660922fe62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"
Sep 15 13:00:00.352737 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.352718    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2dwx\" (UniqueName: \"kubernetes.io/projected/16abf046-395a-4e20-9bd3-74be0b9732c0-kube-api-access-d2dwx\") pod \"maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6\" (UID: \"16abf046-395a-4e20-9bd3-74be0b9732c0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"
Sep 15 13:00:00.353092 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.353072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s75xm\" (UniqueName: \"kubernetes.io/projected/a744131f-9791-45b8-9815-537ab9b389ae-kube-api-access-s75xm\") pod \"maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc\" (UID: \"a744131f-9791-45b8-9815-537ab9b389ae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"
Sep 15 13:00:00.480431 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.480395    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"
Sep 15 13:00:00.489915 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.489890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"
Sep 15 13:00:00.500681 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.500646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"
Sep 15 13:00:00.512421 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.512209    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"
Sep 15 13:00:00.801935 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.801860    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"]
Sep 15 13:00:00.807451 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:00.807423    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d332f41_9e18_445e_8dd9_2b660922fe62.slice/crio-3e7f3d84c3f74182fc510254bcef64066d5c0ac341a79925a772f13d9d329707 WatchSource:0}: Error finding container 3e7f3d84c3f74182fc510254bcef64066d5c0ac341a79925a772f13d9d329707: Status 404 returned error can't find the container with id 3e7f3d84c3f74182fc510254bcef64066d5c0ac341a79925a772f13d9d329707
Sep 15 13:00:00.809529 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.809508    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"]
Sep 15 13:00:00.810628 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:00.810607    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda744131f_9791_45b8_9815_537ab9b389ae.slice/crio-6173305245bff2135f8b54c3587cc3347377f768623475b0a35d0531a769f19a WatchSource:0}: Error finding container 6173305245bff2135f8b54c3587cc3347377f768623475b0a35d0531a769f19a: Status 404 returned error can't find the container with id 6173305245bff2135f8b54c3587cc3347377f768623475b0a35d0531a769f19a
Sep 15 13:00:00.840168 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.840139    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"]
Sep 15 13:00:00.841409 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:00.841383    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16abf046_395a_4e20_9bd3_74be0b9732c0.slice/crio-4006b36c705792423c4288e344b1370daf5bb8cfa2d4bb92e975a3bc5cb9d885 WatchSource:0}: Error finding container 4006b36c705792423c4288e344b1370daf5bb8cfa2d4bb92e975a3bc5cb9d885: Status 404 returned error can't find the container with id 4006b36c705792423c4288e344b1370daf5bb8cfa2d4bb92e975a3bc5cb9d885
Sep 15 13:00:00.842324 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:00.842262    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b2317b_dc8a_4d60_9ec6_9621e2102196.slice/crio-2a3a56cb5289991cf5294138272af5c0ee6d0b06ebd9032f63737b4ec2b65e29 WatchSource:0}: Error finding container 2a3a56cb5289991cf5294138272af5c0ee6d0b06ebd9032f63737b4ec2b65e29: Status 404 returned error can't find the container with id 2a3a56cb5289991cf5294138272af5c0ee6d0b06ebd9032f63737b4ec2b65e29
Sep 15 13:00:00.842958 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:00.842934    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"]
Sep 15 13:00:01.530357 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.530317    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd" event={"ID":"6d332f41-9e18-445e-8dd9-2b660922fe62","Type":"ContainerStarted","Data":"615aa60ced7dad7b47a32af98d52427629ea77e3e9fcd932681f68b760f968bc"}
Sep 15 13:00:01.531093 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.530378    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd" event={"ID":"6d332f41-9e18-445e-8dd9-2b660922fe62","Type":"ContainerStarted","Data":"3e7f3d84c3f74182fc510254bcef64066d5c0ac341a79925a772f13d9d329707"}
Sep 15 13:00:01.532098 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.532065    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv" event={"ID":"35b2317b-dc8a-4d60-9ec6-9621e2102196","Type":"ContainerStarted","Data":"0681e03e2806c74fca47f5c67eadf2e335d0a439b3ed45bd346fd8cc651974a9"}
Sep 15 13:00:01.532228 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.532126    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv" event={"ID":"35b2317b-dc8a-4d60-9ec6-9621e2102196","Type":"ContainerStarted","Data":"2a3a56cb5289991cf5294138272af5c0ee6d0b06ebd9032f63737b4ec2b65e29"}
Sep 15 13:00:01.533712 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.533687    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc" event={"ID":"a744131f-9791-45b8-9815-537ab9b389ae","Type":"ContainerStarted","Data":"36c99385a4e79a57f000587ab131108a9c49e708ec7f6ab62809f2674ba1d00c"}
Sep 15 13:00:01.533837 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.533732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc" event={"ID":"a744131f-9791-45b8-9815-537ab9b389ae","Type":"ContainerStarted","Data":"6173305245bff2135f8b54c3587cc3347377f768623475b0a35d0531a769f19a"}
Sep 15 13:00:01.535366 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.535340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6" event={"ID":"16abf046-395a-4e20-9bd3-74be0b9732c0","Type":"ContainerStarted","Data":"6bc308594d30ecb149190f665f83d7c97f25433c6c994a5850a72b24d4942347"}
Sep 15 13:00:01.535459 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.535377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6" event={"ID":"16abf046-395a-4e20-9bd3-74be0b9732c0","Type":"ContainerStarted","Data":"4006b36c705792423c4288e344b1370daf5bb8cfa2d4bb92e975a3bc5cb9d885"}
Sep 15 13:00:01.551838 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.551794    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd" podStartSLOduration=1.5517793979999999 podStartE2EDuration="1.551779398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:00:01.549114761 +0000 UTC m=+919.341816495" watchObservedRunningTime="2026-09-15 13:00:01.551779398 +0000 UTC m=+919.344481130"
Sep 15 13:00:01.567699 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.567651    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6" podStartSLOduration=1.567637477 podStartE2EDuration="1.567637477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:00:01.566358765 +0000 UTC m=+919.359060495" watchObservedRunningTime="2026-09-15 13:00:01.567637477 +0000 UTC m=+919.360339207"
Sep 15 13:00:01.583984 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.583934    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc" podStartSLOduration=1.583920381 podStartE2EDuration="1.583920381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:00:01.581988552 +0000 UTC m=+919.374690283" watchObservedRunningTime="2026-09-15 13:00:01.583920381 +0000 UTC m=+919.376622112"
Sep 15 13:00:01.599832 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:01.599788    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv" podStartSLOduration=1.599772659 podStartE2EDuration="1.599772659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:00:01.597790378 +0000 UTC m=+919.390492110" watchObservedRunningTime="2026-09-15 13:00:01.599772659 +0000 UTC m=+919.392474390"
Sep 15 13:00:02.544731 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:02.544692    2562 generic.go:356] "Generic (PLEG): container finished" podID="a744131f-9791-45b8-9815-537ab9b389ae" containerID="36c99385a4e79a57f000587ab131108a9c49e708ec7f6ab62809f2674ba1d00c" exitCode=0
Sep 15 13:00:02.545198 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:02.544812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc" event={"ID":"a744131f-9791-45b8-9815-537ab9b389ae","Type":"ContainerDied","Data":"36c99385a4e79a57f000587ab131108a9c49e708ec7f6ab62809f2674ba1d00c"}
Sep 15 13:00:02.546288 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:02.546258    2562 generic.go:356] "Generic (PLEG): container finished" podID="16abf046-395a-4e20-9bd3-74be0b9732c0" containerID="6bc308594d30ecb149190f665f83d7c97f25433c6c994a5850a72b24d4942347" exitCode=0
Sep 15 13:00:02.546402 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:02.546333    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6" event={"ID":"16abf046-395a-4e20-9bd3-74be0b9732c0","Type":"ContainerDied","Data":"6bc308594d30ecb149190f665f83d7c97f25433c6c994a5850a72b24d4942347"}
Sep 15 13:00:02.547811 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:02.547788    2562 generic.go:356] "Generic (PLEG): container finished" podID="6d332f41-9e18-445e-8dd9-2b660922fe62" containerID="615aa60ced7dad7b47a32af98d52427629ea77e3e9fcd932681f68b760f968bc" exitCode=0
Sep 15 13:00:02.547932 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:02.547848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd" event={"ID":"6d332f41-9e18-445e-8dd9-2b660922fe62","Type":"ContainerDied","Data":"615aa60ced7dad7b47a32af98d52427629ea77e3e9fcd932681f68b760f968bc"}
Sep 15 13:00:02.549376 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:02.549358    2562 generic.go:356] "Generic (PLEG): container finished" podID="35b2317b-dc8a-4d60-9ec6-9621e2102196" containerID="0681e03e2806c74fca47f5c67eadf2e335d0a439b3ed45bd346fd8cc651974a9" exitCode=0
Sep 15 13:00:02.549456 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:02.549381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv" event={"ID":"35b2317b-dc8a-4d60-9ec6-9621e2102196","Type":"ContainerDied","Data":"0681e03e2806c74fca47f5c67eadf2e335d0a439b3ed45bd346fd8cc651974a9"}
Sep 15 13:00:03.748782 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.748492    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"
Sep 15 13:00:03.784003 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.783970    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s75xm\" (UniqueName: \"kubernetes.io/projected/a744131f-9791-45b8-9815-537ab9b389ae-kube-api-access-s75xm\") pod \"a744131f-9791-45b8-9815-537ab9b389ae\" (UID: \"a744131f-9791-45b8-9815-537ab9b389ae\") "
Sep 15 13:00:03.784182 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.784100    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"
Sep 15 13:00:03.788469 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.788425    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a744131f-9791-45b8-9815-537ab9b389ae-kube-api-access-s75xm" (OuterVolumeSpecName: "kube-api-access-s75xm") pod "a744131f-9791-45b8-9815-537ab9b389ae" (UID: "a744131f-9791-45b8-9815-537ab9b389ae"). InnerVolumeSpecName "kube-api-access-s75xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:03.819258 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.819230    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"
Sep 15 13:00:03.824639 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.824605    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"
Sep 15 13:00:03.885317 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.885291    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4d9l\" (UniqueName: \"kubernetes.io/projected/6d332f41-9e18-445e-8dd9-2b660922fe62-kube-api-access-r4d9l\") pod \"6d332f41-9e18-445e-8dd9-2b660922fe62\" (UID: \"6d332f41-9e18-445e-8dd9-2b660922fe62\") "
Sep 15 13:00:03.885464 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.885428    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2fjc\" (UniqueName: \"kubernetes.io/projected/35b2317b-dc8a-4d60-9ec6-9621e2102196-kube-api-access-l2fjc\") pod \"35b2317b-dc8a-4d60-9ec6-9621e2102196\" (UID: \"35b2317b-dc8a-4d60-9ec6-9621e2102196\") "
Sep 15 13:00:03.885529 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.885474    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2dwx\" (UniqueName: \"kubernetes.io/projected/16abf046-395a-4e20-9bd3-74be0b9732c0-kube-api-access-d2dwx\") pod \"16abf046-395a-4e20-9bd3-74be0b9732c0\" (UID: \"16abf046-395a-4e20-9bd3-74be0b9732c0\") "
Sep 15 13:00:03.885666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.885652    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s75xm\" (UniqueName: \"kubernetes.io/projected/a744131f-9791-45b8-9815-537ab9b389ae-kube-api-access-s75xm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:03.887323 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.887297    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d332f41-9e18-445e-8dd9-2b660922fe62-kube-api-access-r4d9l" (OuterVolumeSpecName: "kube-api-access-r4d9l") pod "6d332f41-9e18-445e-8dd9-2b660922fe62" (UID: "6d332f41-9e18-445e-8dd9-2b660922fe62"). InnerVolumeSpecName "kube-api-access-r4d9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:03.887428 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.887393    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b2317b-dc8a-4d60-9ec6-9621e2102196-kube-api-access-l2fjc" (OuterVolumeSpecName: "kube-api-access-l2fjc") pod "35b2317b-dc8a-4d60-9ec6-9621e2102196" (UID: "35b2317b-dc8a-4d60-9ec6-9621e2102196"). InnerVolumeSpecName "kube-api-access-l2fjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:03.887474 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.887442    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16abf046-395a-4e20-9bd3-74be0b9732c0-kube-api-access-d2dwx" (OuterVolumeSpecName: "kube-api-access-d2dwx") pod "16abf046-395a-4e20-9bd3-74be0b9732c0" (UID: "16abf046-395a-4e20-9bd3-74be0b9732c0"). InnerVolumeSpecName "kube-api-access-d2dwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:03.986543 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.986500    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2fjc\" (UniqueName: \"kubernetes.io/projected/35b2317b-dc8a-4d60-9ec6-9621e2102196-kube-api-access-l2fjc\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:03.986543 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.986538    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2dwx\" (UniqueName: \"kubernetes.io/projected/16abf046-395a-4e20-9bd3-74be0b9732c0-kube-api-access-d2dwx\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:03.986543 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:03.986549    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4d9l\" (UniqueName: \"kubernetes.io/projected/6d332f41-9e18-445e-8dd9-2b660922fe62-kube-api-access-r4d9l\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:04.562373 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.562343    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc"
Sep 15 13:00:04.562554 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.562351    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc" event={"ID":"a744131f-9791-45b8-9815-537ab9b389ae","Type":"ContainerDied","Data":"6173305245bff2135f8b54c3587cc3347377f768623475b0a35d0531a769f19a"}
Sep 15 13:00:04.562554 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.562479    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6173305245bff2135f8b54c3587cc3347377f768623475b0a35d0531a769f19a"
Sep 15 13:00:04.563739 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.563717    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"
Sep 15 13:00:04.563852 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.563753    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6" event={"ID":"16abf046-395a-4e20-9bd3-74be0b9732c0","Type":"ContainerDied","Data":"4006b36c705792423c4288e344b1370daf5bb8cfa2d4bb92e975a3bc5cb9d885"}
Sep 15 13:00:04.563852 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.563811    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4006b36c705792423c4288e344b1370daf5bb8cfa2d4bb92e975a3bc5cb9d885"
Sep 15 13:00:04.565007 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.564986    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"
Sep 15 13:00:04.565146 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.565001    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd" event={"ID":"6d332f41-9e18-445e-8dd9-2b660922fe62","Type":"ContainerDied","Data":"3e7f3d84c3f74182fc510254bcef64066d5c0ac341a79925a772f13d9d329707"}
Sep 15 13:00:04.565146 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.565067    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e7f3d84c3f74182fc510254bcef64066d5c0ac341a79925a772f13d9d329707"
Sep 15 13:00:04.566390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.566376    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv"
Sep 15 13:00:04.566472 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.566371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv" event={"ID":"35b2317b-dc8a-4d60-9ec6-9621e2102196","Type":"ContainerDied","Data":"2a3a56cb5289991cf5294138272af5c0ee6d0b06ebd9032f63737b4ec2b65e29"}
Sep 15 13:00:04.566522 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:04.566494    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a3a56cb5289991cf5294138272af5c0ee6d0b06ebd9032f63737b4ec2b65e29"
Sep 15 13:00:07.518082 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518023    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"]
Sep 15 13:00:07.518531 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518517    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a744131f-9791-45b8-9815-537ab9b389ae" containerName="cleanup"
Sep 15 13:00:07.518577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518534    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a744131f-9791-45b8-9815-537ab9b389ae" containerName="cleanup"
Sep 15 13:00:07.518577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518546    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d332f41-9e18-445e-8dd9-2b660922fe62" containerName="cleanup"
Sep 15 13:00:07.518577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518552    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d332f41-9e18-445e-8dd9-2b660922fe62" containerName="cleanup"
Sep 15 13:00:07.518577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518569    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35b2317b-dc8a-4d60-9ec6-9621e2102196" containerName="cleanup"
Sep 15 13:00:07.518577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518575    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="35b2317b-dc8a-4d60-9ec6-9621e2102196" containerName="cleanup"
Sep 15 13:00:07.518729 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518581    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16abf046-395a-4e20-9bd3-74be0b9732c0" containerName="cleanup"
Sep 15 13:00:07.518729 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518586    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="16abf046-395a-4e20-9bd3-74be0b9732c0" containerName="cleanup"
Sep 15 13:00:07.518729 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518660    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d332f41-9e18-445e-8dd9-2b660922fe62" containerName="cleanup"
Sep 15 13:00:07.518729 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518668    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="16abf046-395a-4e20-9bd3-74be0b9732c0" containerName="cleanup"
Sep 15 13:00:07.518729 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518674    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a744131f-9791-45b8-9815-537ab9b389ae" containerName="cleanup"
Sep 15 13:00:07.518729 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.518685    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="35b2317b-dc8a-4d60-9ec6-9621e2102196" containerName="cleanup"
Sep 15 13:00:07.521112 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.521090    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:07.524625 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.524600    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:00:07.524803 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.524778    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:00:07.525912 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.525895    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:00:07.532660 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.532635    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"]
Sep 15 13:00:07.648962 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.648906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74wxq\" (UniqueName: \"kubernetes.io/projected/8e889837-8e8a-420c-9de4-3442ada6828f-kube-api-access-74wxq\") pod \"maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f\" (UID: \"8e889837-8e8a-420c-9de4-3442ada6828f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:07.649178 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.649058    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e889837-8e8a-420c-9de4-3442ada6828f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f\" (UID: \"8e889837-8e8a-420c-9de4-3442ada6828f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:07.750531 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.750493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e889837-8e8a-420c-9de4-3442ada6828f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f\" (UID: \"8e889837-8e8a-420c-9de4-3442ada6828f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:07.750707 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.750618    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74wxq\" (UniqueName: \"kubernetes.io/projected/8e889837-8e8a-420c-9de4-3442ada6828f-kube-api-access-74wxq\") pod \"maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f\" (UID: \"8e889837-8e8a-420c-9de4-3442ada6828f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:07.751355 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.751332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e889837-8e8a-420c-9de4-3442ada6828f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f\" (UID: \"8e889837-8e8a-420c-9de4-3442ada6828f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:07.759645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.759618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74wxq\" (UniqueName: \"kubernetes.io/projected/8e889837-8e8a-420c-9de4-3442ada6828f-kube-api-access-74wxq\") pod \"maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f\" (UID: \"8e889837-8e8a-420c-9de4-3442ada6828f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:07.847090 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:07.846975    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:08.126618 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:08.126528    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"]
Sep 15 13:00:08.150750 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:08.150700    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e889837_8e8a_420c_9de4_3442ada6828f.slice/crio-fa2dd5cdaa81e33e55bc76865e4b84f7c173cfe286422e8a81ffab698294c44d WatchSource:0}: Error finding container fa2dd5cdaa81e33e55bc76865e4b84f7c173cfe286422e8a81ffab698294c44d: Status 404 returned error can't find the container with id fa2dd5cdaa81e33e55bc76865e4b84f7c173cfe286422e8a81ffab698294c44d
Sep 15 13:00:08.595682 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:08.595627    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f" event={"ID":"8e889837-8e8a-420c-9de4-3442ada6828f","Type":"ContainerStarted","Data":"f885a454e6d83224f5fd8c5319af2aa231b264f9079912e2cafcf2d99acd933b"}
Sep 15 13:00:08.596286 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:08.595756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f" event={"ID":"8e889837-8e8a-420c-9de4-3442ada6828f","Type":"ContainerStarted","Data":"fa2dd5cdaa81e33e55bc76865e4b84f7c173cfe286422e8a81ffab698294c44d"}
Sep 15 13:00:08.615309 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:08.615260    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f" podStartSLOduration=1.6152437339999999 podStartE2EDuration="1.615243734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:00:08.613060688 +0000 UTC m=+926.405762423" watchObservedRunningTime="2026-09-15 13:00:08.615243734 +0000 UTC m=+926.407945465"
Sep 15 13:00:09.602409 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:09.602375    2562 generic.go:356] "Generic (PLEG): container finished" podID="8e889837-8e8a-420c-9de4-3442ada6828f" containerID="f885a454e6d83224f5fd8c5319af2aa231b264f9079912e2cafcf2d99acd933b" exitCode=0
Sep 15 13:00:09.602778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:09.602469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f" event={"ID":"8e889837-8e8a-420c-9de4-3442ada6828f","Type":"ContainerDied","Data":"f885a454e6d83224f5fd8c5319af2aa231b264f9079912e2cafcf2d99acd933b"}
Sep 15 13:00:10.783120 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:10.783097    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:10.887419 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:10.887330    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e889837-8e8a-420c-9de4-3442ada6828f-maas-api-service-ca\") pod \"8e889837-8e8a-420c-9de4-3442ada6828f\" (UID: \"8e889837-8e8a-420c-9de4-3442ada6828f\") "
Sep 15 13:00:10.887577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:10.887560    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74wxq\" (UniqueName: \"kubernetes.io/projected/8e889837-8e8a-420c-9de4-3442ada6828f-kube-api-access-74wxq\") pod \"8e889837-8e8a-420c-9de4-3442ada6828f\" (UID: \"8e889837-8e8a-420c-9de4-3442ada6828f\") "
Sep 15 13:00:10.887718 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:10.887694    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e889837-8e8a-420c-9de4-3442ada6828f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e889837-8e8a-420c-9de4-3442ada6828f" (UID: "8e889837-8e8a-420c-9de4-3442ada6828f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:00:10.888186 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:10.888162    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e889837-8e8a-420c-9de4-3442ada6828f-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:10.889616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:10.889590    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e889837-8e8a-420c-9de4-3442ada6828f-kube-api-access-74wxq" (OuterVolumeSpecName: "kube-api-access-74wxq") pod "8e889837-8e8a-420c-9de4-3442ada6828f" (UID: "8e889837-8e8a-420c-9de4-3442ada6828f"). InnerVolumeSpecName "kube-api-access-74wxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:10.988758 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:10.988725    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74wxq\" (UniqueName: \"kubernetes.io/projected/8e889837-8e8a-420c-9de4-3442ada6828f-kube-api-access-74wxq\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:11.615403 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:11.615372    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"
Sep 15 13:00:11.615552 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:11.615413    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f" event={"ID":"8e889837-8e8a-420c-9de4-3442ada6828f","Type":"ContainerDied","Data":"fa2dd5cdaa81e33e55bc76865e4b84f7c173cfe286422e8a81ffab698294c44d"}
Sep 15 13:00:11.615552 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:11.615459    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa2dd5cdaa81e33e55bc76865e4b84f7c173cfe286422e8a81ffab698294c44d"
Sep 15 13:00:13.436219 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:13.436175    2562 patch_prober.go:28] interesting pod/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 15 13:00:13.436624 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:13.436278    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 15 13:00:14.216197 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:14.216162    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"]
Sep 15 13:00:14.229675 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:14.229649    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-dc842c6b-29824620-9hvjd"]
Sep 15 13:00:14.626572 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:14.626491    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"]
Sep 15 13:00:14.629728 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:14.629695    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc842c6b-bzw5f"]
Sep 15 13:00:14.758077 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:14.757996    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d332f41-9e18-445e-8dd9-2b660922fe62" path="/var/lib/kubelet/pods/6d332f41-9e18-445e-8dd9-2b660922fe62/volumes"
Sep 15 13:00:14.758511 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:14.758489    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e889837-8e8a-420c-9de4-3442ada6828f" path="/var/lib/kubelet/pods/8e889837-8e8a-420c-9de4-3442ada6828f/volumes"
Sep 15 13:00:15.642161 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:15.642079    2562 generic.go:356] "Generic (PLEG): container finished" podID="78225ab2-03b3-4575-8231-c81f59e5918e" containerID="7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3" exitCode=0
Sep 15 13:00:15.642553 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:15.642152    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" event={"ID":"78225ab2-03b3-4575-8231-c81f59e5918e","Type":"ContainerDied","Data":"7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3"}
Sep 15 13:00:15.642599 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:15.642562    2562 scope.go:117] "RemoveContainer" containerID="7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3"
Sep 15 13:00:15.643659 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:15.643623    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:15.643728 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:15.643697    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:15.643770 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:15.643734    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:15.723119 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:15.723014    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"]
Sep 15 13:00:15.786107 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:15.786070    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:00:16.553488 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.553455    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"]
Sep 15 13:00:16.554014 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.553995    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e889837-8e8a-420c-9de4-3442ada6828f" containerName="revoke-keys"
Sep 15 13:00:16.554149 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.554015    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e889837-8e8a-420c-9de4-3442ada6828f" containerName="revoke-keys"
Sep 15 13:00:16.554149 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.554133    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e889837-8e8a-420c-9de4-3442ada6828f" containerName="revoke-keys"
Sep 15 13:00:16.556372 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.556351    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.559211 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.559181    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-179a5f99-openshift-default-dockercfg-rsnjq\""
Sep 15 13:00:16.572232 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.572208    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"]
Sep 15 13:00:16.651491 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.651450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" event={"ID":"78225ab2-03b3-4575-8231-c81f59e5918e","Type":"ContainerStarted","Data":"0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e"}
Sep 15 13:00:16.651923 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.651504    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" containerName="istio-proxy" containerID="cri-o://0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e" gracePeriod=30
Sep 15 13:00:16.655287 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655204    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-certs\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.655287 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c188d78-bfd9-439a-ae26-73858012826e-istio-podinfo\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.655287 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-data\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.655601 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq2q4\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-kube-api-access-mq2q4\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.655601 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655404    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-credential-socket\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.655601 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655496    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c188d78-bfd9-439a-ae26-73858012826e-istiod-ca-cert\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.655601 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-envoy\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.655816 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-istio-token\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.655816 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.655704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-socket\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756513 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-istio-token\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756674 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-socket\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756674 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-certs\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756674 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c188d78-bfd9-439a-ae26-73858012826e-istio-podinfo\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756674 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-data\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756674 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756618    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq2q4\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-kube-api-access-mq2q4\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756674 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-credential-socket\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756993 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756696    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c188d78-bfd9-439a-ae26-73858012826e-istiod-ca-cert\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.756993 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-envoy\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.757099 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.756991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-certs\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.757099 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.757072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-socket\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.757182 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.757112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-data\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.757342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.757325    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-credential-socket\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.757486 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.757461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c188d78-bfd9-439a-ae26-73858012826e-istiod-ca-cert\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.759083 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.759050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-envoy\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.759289 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.759271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c188d78-bfd9-439a-ae26-73858012826e-istio-podinfo\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.764552 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.764533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-istio-token\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.765025 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.765005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq2q4\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-kube-api-access-mq2q4\") pod \"e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") " pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:16.867135 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:16.867021    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:17.154280 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:17.154255    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"]
Sep 15 13:00:17.155858 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:17.155826    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c188d78_bfd9_439a_ae26_73858012826e.slice/crio-4db931955cbdf0c43d691f7cc6c5da3a135f2d9a107be959cd62892b2b5fd534 WatchSource:0}: Error finding container 4db931955cbdf0c43d691f7cc6c5da3a135f2d9a107be959cd62892b2b5fd534: Status 404 returned error can't find the container with id 4db931955cbdf0c43d691f7cc6c5da3a135f2d9a107be959cd62892b2b5fd534
Sep 15 13:00:17.160049 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:17.159780    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:17.160049 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:17.159894    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:17.160049 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:17.159954    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:17.662686 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:17.662647    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" event={"ID":"8c188d78-bfd9-439a-ae26-73858012826e","Type":"ContainerStarted","Data":"7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342"}
Sep 15 13:00:17.663291 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:17.663265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" event={"ID":"8c188d78-bfd9-439a-ae26-73858012826e","Type":"ContainerStarted","Data":"4db931955cbdf0c43d691f7cc6c5da3a135f2d9a107be959cd62892b2b5fd534"}
Sep 15 13:00:17.688852 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:17.688803    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" podStartSLOduration=1.6887879 podStartE2EDuration="1.6887879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:00:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:00:17.686697592 +0000 UTC m=+935.479399318" watchObservedRunningTime="2026-09-15 13:00:17.6887879 +0000 UTC m=+935.481489699"
Sep 15 13:00:17.867459 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:17.867430    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:18.873203 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:18.873174    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:19.674645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:19.674616    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:19.675702 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:19.675678    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:21.913200 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:21.913177    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 13:00:22.015241 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015158    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78225ab2-03b3-4575-8231-c81f59e5918e-istiod-ca-cert\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015241 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015196    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-data\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015269    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-istio-token\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015292    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-socket\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015324    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-envoy\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015357    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78225ab2-03b3-4575-8231-c81f59e5918e-istio-podinfo\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015382    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xmmb\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-kube-api-access-2xmmb\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015416    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-credential-socket\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015780 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015479    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-certs\") pod \"78225ab2-03b3-4575-8231-c81f59e5918e\" (UID: \"78225ab2-03b3-4575-8231-c81f59e5918e\") "
Sep 15 13:00:22.015780 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015531    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-data" (OuterVolumeSpecName: "istio-data") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:22.015780 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015627    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:22.015780 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015657    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78225ab2-03b3-4575-8231-c81f59e5918e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:00:22.015994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015791    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:22.015994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015848    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:22.015994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015974    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.015994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.015991    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.016228 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.016000    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.016228 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.016009    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78225ab2-03b3-4575-8231-c81f59e5918e-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.016228 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.016017    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.017729 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.017704    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78225ab2-03b3-4575-8231-c81f59e5918e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:00:22.017827 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.017725    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-istio-token" (OuterVolumeSpecName: "istio-token") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:22.017999 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.017980    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-kube-api-access-2xmmb" (OuterVolumeSpecName: "kube-api-access-2xmmb") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "kube-api-access-2xmmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:22.018084 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.017994    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78225ab2-03b3-4575-8231-c81f59e5918e" (UID: "78225ab2-03b3-4575-8231-c81f59e5918e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:22.117290 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.117257    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.117290 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.117285    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78225ab2-03b3-4575-8231-c81f59e5918e-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.117290 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.117294    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78225ab2-03b3-4575-8231-c81f59e5918e-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.117512 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.117303    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xmmb\" (UniqueName: \"kubernetes.io/projected/78225ab2-03b3-4575-8231-c81f59e5918e-kube-api-access-2xmmb\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:22.698643 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.698611    2562 generic.go:356] "Generic (PLEG): container finished" podID="78225ab2-03b3-4575-8231-c81f59e5918e" containerID="0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e" exitCode=0
Sep 15 13:00:22.698824 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.698655    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" event={"ID":"78225ab2-03b3-4575-8231-c81f59e5918e","Type":"ContainerDied","Data":"0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e"}
Sep 15 13:00:22.698824 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.698662    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"
Sep 15 13:00:22.698824 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.698698    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6" event={"ID":"78225ab2-03b3-4575-8231-c81f59e5918e","Type":"ContainerDied","Data":"f6d089e7ad7b6968054f32c9a09674ee17fd79c2646a55cd4659c9c6b366f601"}
Sep 15 13:00:22.698824 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.698723    2562 scope.go:117] "RemoveContainer" containerID="0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e"
Sep 15 13:00:22.709203 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.709185    2562 scope.go:117] "RemoveContainer" containerID="7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3"
Sep 15 13:00:22.717426 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.717407    2562 scope.go:117] "RemoveContainer" containerID="0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e"
Sep 15 13:00:22.717680 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:00:22.717662    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e\": container with ID starting with 0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e not found: ID does not exist" containerID="0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e"
Sep 15 13:00:22.717721 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.717689    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e"} err="failed to get container status \"0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e\": rpc error: code = NotFound desc = could not find container \"0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e\": container with ID starting with 0eb4c3333b36773d809d05834c5dbeafb60fcbb7f8a4859c351de5219671c83e not found: ID does not exist"
Sep 15 13:00:22.717721 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.717708    2562 scope.go:117] "RemoveContainer" containerID="7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3"
Sep 15 13:00:22.717942 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:00:22.717927    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3\": container with ID starting with 7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3 not found: ID does not exist" containerID="7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3"
Sep 15 13:00:22.717981 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.717946    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3"} err="failed to get container status \"7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3\": rpc error: code = NotFound desc = could not find container \"7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3\": container with ID starting with 7946f447023a0a8386cd7ee2a4256fb2279270d723f1f12345b2775ac0a9d3b3 not found: ID does not exist"
Sep 15 13:00:22.723857 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.723830    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"]
Sep 15 13:00:22.728243 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.728224    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dc842c6b-openshift-default-7d6fc86d8-89vj6"]
Sep 15 13:00:22.756518 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:22.756493    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" path="/var/lib/kubelet/pods/78225ab2-03b3-4575-8231-c81f59e5918e/volumes"
Sep 15 13:00:34.686559 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:34.686523    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:00:40.988745 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.988700    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"]
Sep 15 13:00:40.989252 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.989239    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" containerName="istio-proxy"
Sep 15 13:00:40.989252 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.989254    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" containerName="istio-proxy"
Sep 15 13:00:40.989353 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.989265    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" containerName="istio-proxy"
Sep 15 13:00:40.989353 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.989270    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" containerName="istio-proxy"
Sep 15 13:00:40.989353 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.989348    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" containerName="istio-proxy"
Sep 15 13:00:40.989449 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.989363    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="78225ab2-03b3-4575-8231-c81f59e5918e" containerName="istio-proxy"
Sep 15 13:00:40.992408 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.992391    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:40.994797 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.994778    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-528f6b\"/\"kube-root-ca.crt\""
Sep 15 13:00:40.995813 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.995791    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-528f6b\"/\"test-model-528f6b-kserve-self-signed-certs\""
Sep 15 13:00:40.995947 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.995843    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-528f6b\"/\"default-dockercfg-5sfn4\""
Sep 15 13:00:40.995947 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:40.995795    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-528f6b\"/\"openshift-service-ca.crt\""
Sep 15 13:00:41.001930 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.001905    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"]
Sep 15 13:00:41.143685 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.143646    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-model-cache\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.144021 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.144001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-tmp-dir\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.144220 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.144202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nw6s\" (UniqueName: \"kubernetes.io/projected/f6f971ee-83ba-4a5e-b370-58488310ea21-kube-api-access-8nw6s\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.144376 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.144358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-home\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.144532 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.144518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f971ee-83ba-4a5e-b370-58488310ea21-tls-certs\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.144687 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.144673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-dshm\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.246421 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.246314    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nw6s\" (UniqueName: \"kubernetes.io/projected/f6f971ee-83ba-4a5e-b370-58488310ea21-kube-api-access-8nw6s\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.246421 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.246388    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-home\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.246645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.246443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f971ee-83ba-4a5e-b370-58488310ea21-tls-certs\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.246645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.246491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-dshm\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.246645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.246581    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-model-cache\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.246801 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.246659    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-tmp-dir\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.247182 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.247147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-tmp-dir\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.247586 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.247562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-model-cache\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.248343 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.248319    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-home\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.251394 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.251373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f971ee-83ba-4a5e-b370-58488310ea21-tls-certs\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.253185 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.253165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-dshm\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.263658 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.263618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nw6s\" (UniqueName: \"kubernetes.io/projected/f6f971ee-83ba-4a5e-b370-58488310ea21-kube-api-access-8nw6s\") pod \"test-model-528f6b-kserve-6d7f7774c5-whpnm\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") " pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.312471 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.312412    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:41.606595 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.606569    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"]
Sep 15 13:00:41.607971 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:41.607948    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6f971ee_83ba_4a5e_b370_58488310ea21.slice/crio-ea81846005329cfdc3eb33f4115e7fbb092c7c53350390fd7ecc78b4d2738103 WatchSource:0}: Error finding container ea81846005329cfdc3eb33f4115e7fbb092c7c53350390fd7ecc78b4d2738103: Status 404 returned error can't find the container with id ea81846005329cfdc3eb33f4115e7fbb092c7c53350390fd7ecc78b4d2738103
Sep 15 13:00:41.818640 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.818604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm" event={"ID":"f6f971ee-83ba-4a5e-b370-58488310ea21","Type":"ContainerStarted","Data":"4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538"}
Sep 15 13:00:41.818640 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:41.818644    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm" event={"ID":"f6f971ee-83ba-4a5e-b370-58488310ea21","Type":"ContainerStarted","Data":"ea81846005329cfdc3eb33f4115e7fbb092c7c53350390fd7ecc78b4d2738103"}
Sep 15 13:00:43.942243 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.942179    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm" podStartSLOduration=3.942154122 podStartE2EDuration="3.942154122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:00:41.841220547 +0000 UTC m=+959.633922279" watchObservedRunningTime="2026-09-15 13:00:43.942154122 +0000 UTC m=+961.734855857"
Sep 15 13:00:43.944911 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.944884    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"]
Sep 15 13:00:43.950397 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.950377    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:43.952954 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.952925    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:00:43.953871 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.953847    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:00:43.954188 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.953870    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:00:43.958132 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.957983    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"]
Sep 15 13:00:43.976468 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.976436    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef63b2aa-fdea-45bf-9880-4612bac2cda7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-179a5f99-99bb2\" (UID: \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:43.976593 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:43.976555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdr6t\" (UniqueName: \"kubernetes.io/projected/ef63b2aa-fdea-45bf-9880-4612bac2cda7-kube-api-access-sdr6t\") pod \"maas-api-revoke-keys-e2e-derive-179a5f99-99bb2\" (UID: \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:44.077311 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.077278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef63b2aa-fdea-45bf-9880-4612bac2cda7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-179a5f99-99bb2\" (UID: \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:44.077516 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.077348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdr6t\" (UniqueName: \"kubernetes.io/projected/ef63b2aa-fdea-45bf-9880-4612bac2cda7-kube-api-access-sdr6t\") pod \"maas-api-revoke-keys-e2e-derive-179a5f99-99bb2\" (UID: \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:44.077981 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.077952    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef63b2aa-fdea-45bf-9880-4612bac2cda7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-179a5f99-99bb2\" (UID: \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:44.086913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.086889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdr6t\" (UniqueName: \"kubernetes.io/projected/ef63b2aa-fdea-45bf-9880-4612bac2cda7-kube-api-access-sdr6t\") pod \"maas-api-revoke-keys-e2e-derive-179a5f99-99bb2\" (UID: \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:44.265085 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.265000    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:44.551058 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.551019    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"]
Sep 15 13:00:44.553725 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:44.553694    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef63b2aa_fdea_45bf_9880_4612bac2cda7.slice/crio-e39cadbb8344ee73d2ff9f36f2094e9799bf219784059026b98a6ff8f1bbf647 WatchSource:0}: Error finding container e39cadbb8344ee73d2ff9f36f2094e9799bf219784059026b98a6ff8f1bbf647: Status 404 returned error can't find the container with id e39cadbb8344ee73d2ff9f36f2094e9799bf219784059026b98a6ff8f1bbf647
Sep 15 13:00:44.844608 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.844579    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2" event={"ID":"ef63b2aa-fdea-45bf-9880-4612bac2cda7","Type":"ContainerStarted","Data":"d8e4ab55efc822fefe6200ed3480122c895bc06fd82212c242eaa4dc4e8f9b08"}
Sep 15 13:00:44.844742 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.844629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2" event={"ID":"ef63b2aa-fdea-45bf-9880-4612bac2cda7","Type":"ContainerStarted","Data":"e39cadbb8344ee73d2ff9f36f2094e9799bf219784059026b98a6ff8f1bbf647"}
Sep 15 13:00:44.867559 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:44.867491    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2" podStartSLOduration=1.867468422 podStartE2EDuration="1.867468422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:00:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:00:44.865772897 +0000 UTC m=+962.658474629" watchObservedRunningTime="2026-09-15 13:00:44.867468422 +0000 UTC m=+962.660170153"
Sep 15 13:00:45.852591 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:45.852553    2562 generic.go:356] "Generic (PLEG): container finished" podID="ef63b2aa-fdea-45bf-9880-4612bac2cda7" containerID="d8e4ab55efc822fefe6200ed3480122c895bc06fd82212c242eaa4dc4e8f9b08" exitCode=0
Sep 15 13:00:45.853089 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:45.852633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2" event={"ID":"ef63b2aa-fdea-45bf-9880-4612bac2cda7","Type":"ContainerDied","Data":"d8e4ab55efc822fefe6200ed3480122c895bc06fd82212c242eaa4dc4e8f9b08"}
Sep 15 13:00:45.854438 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:45.854413    2562 generic.go:356] "Generic (PLEG): container finished" podID="8c188d78-bfd9-439a-ae26-73858012826e" containerID="7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342" exitCode=0
Sep 15 13:00:45.854545 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:45.854489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" event={"ID":"8c188d78-bfd9-439a-ae26-73858012826e","Type":"ContainerDied","Data":"7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342"}
Sep 15 13:00:45.854974 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:45.854958    2562 scope.go:117] "RemoveContainer" containerID="7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342"
Sep 15 13:00:45.856382 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:45.856352    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:45.856471 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:45.856438    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:45.856514 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:45.856485    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:00:46.863263 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:46.863224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" event={"ID":"8c188d78-bfd9-439a-ae26-73858012826e","Type":"ContainerStarted","Data":"b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99"}
Sep 15 13:00:46.867563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:46.867537    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:46.868707 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:46.868682    2562 patch_prober.go:28] interesting pod/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 13:00:46.868773 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:46.868739    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 13:00:47.017562 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.017541    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:47.112868 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.112837    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdr6t\" (UniqueName: \"kubernetes.io/projected/ef63b2aa-fdea-45bf-9880-4612bac2cda7-kube-api-access-sdr6t\") pod \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\" (UID: \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\") "
Sep 15 13:00:47.113050 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.112921    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef63b2aa-fdea-45bf-9880-4612bac2cda7-maas-api-service-ca\") pod \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\" (UID: \"ef63b2aa-fdea-45bf-9880-4612bac2cda7\") "
Sep 15 13:00:47.113239 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.113215    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef63b2aa-fdea-45bf-9880-4612bac2cda7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef63b2aa-fdea-45bf-9880-4612bac2cda7" (UID: "ef63b2aa-fdea-45bf-9880-4612bac2cda7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:00:47.115139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.115072    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef63b2aa-fdea-45bf-9880-4612bac2cda7-kube-api-access-sdr6t" (OuterVolumeSpecName: "kube-api-access-sdr6t") pod "ef63b2aa-fdea-45bf-9880-4612bac2cda7" (UID: "ef63b2aa-fdea-45bf-9880-4612bac2cda7"). InnerVolumeSpecName "kube-api-access-sdr6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:47.214684 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.214649    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdr6t\" (UniqueName: \"kubernetes.io/projected/ef63b2aa-fdea-45bf-9880-4612bac2cda7-kube-api-access-sdr6t\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:47.214684 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.214679    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef63b2aa-fdea-45bf-9880-4612bac2cda7-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:47.867354 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.867319    2562 patch_prober.go:28] interesting pod/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 13:00:47.867795 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.867384    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 13:00:47.870978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.870954    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"
Sep 15 13:00:47.870978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.870971    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2" event={"ID":"ef63b2aa-fdea-45bf-9880-4612bac2cda7","Type":"ContainerDied","Data":"e39cadbb8344ee73d2ff9f36f2094e9799bf219784059026b98a6ff8f1bbf647"}
Sep 15 13:00:47.871162 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:47.871013    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e39cadbb8344ee73d2ff9f36f2094e9799bf219784059026b98a6ff8f1bbf647"
Sep 15 13:00:48.001899 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.001863    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"]
Sep 15 13:00:48.002362 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.002349    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef63b2aa-fdea-45bf-9880-4612bac2cda7" containerName="revoke-keys"
Sep 15 13:00:48.002414 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.002365    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef63b2aa-fdea-45bf-9880-4612bac2cda7" containerName="revoke-keys"
Sep 15 13:00:48.002465 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.002455    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef63b2aa-fdea-45bf-9880-4612bac2cda7" containerName="revoke-keys"
Sep 15 13:00:48.004677 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.004659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:48.007141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.007119    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:00:48.007390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.007374    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:00:48.007496 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.007391    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:00:48.014343 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.014310    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"]
Sep 15 13:00:48.128112 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.126275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zjtv\" (UniqueName: \"kubernetes.io/projected/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-kube-api-access-9zjtv\") pod \"maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8\" (UID: \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:48.128112 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.126327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8\" (UID: \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:48.228802 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.227477    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zjtv\" (UniqueName: \"kubernetes.io/projected/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-kube-api-access-9zjtv\") pod \"maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8\" (UID: \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:48.228802 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.227532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8\" (UID: \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:48.228802 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.228493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8\" (UID: \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:48.238402 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.238353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zjtv\" (UniqueName: \"kubernetes.io/projected/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-kube-api-access-9zjtv\") pod \"maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8\" (UID: \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:48.315800 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.315775    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:48.609751 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.609725    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"]
Sep 15 13:00:48.611727 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:00:48.611698    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod828a9ef9_3d86_4f92_8632_b6377e0fb1b1.slice/crio-bdc42d391302809e1bba57e56f4a992dc4acb4843616150c79fbb7cee53e4e9f WatchSource:0}: Error finding container bdc42d391302809e1bba57e56f4a992dc4acb4843616150c79fbb7cee53e4e9f: Status 404 returned error can't find the container with id bdc42d391302809e1bba57e56f4a992dc4acb4843616150c79fbb7cee53e4e9f
Sep 15 13:00:48.867619 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.867583    2562 patch_prober.go:28] interesting pod/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 13:00:48.868015 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.867654    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 13:00:48.883880 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:48.883842    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8" event={"ID":"828a9ef9-3d86-4f92-8632-b6377e0fb1b1","Type":"ContainerStarted","Data":"bdc42d391302809e1bba57e56f4a992dc4acb4843616150c79fbb7cee53e4e9f"}
Sep 15 13:00:49.674771 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:49.674741    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:49.867936 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:49.867796    2562 patch_prober.go:28] interesting pod/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 13:00:49.867936 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:49.867866    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 13:00:49.891084 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:49.891009    2562 generic.go:356] "Generic (PLEG): container finished" podID="828a9ef9-3d86-4f92-8632-b6377e0fb1b1" containerID="51b5236c8d8db4de9f0b4a2aab8d8d329cc306436df54e9b062217cb2b8f8935" exitCode=0
Sep 15 13:00:49.891084 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:49.891064    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8" event={"ID":"828a9ef9-3d86-4f92-8632-b6377e0fb1b1","Type":"ContainerDied","Data":"51b5236c8d8db4de9f0b4a2aab8d8d329cc306436df54e9b062217cb2b8f8935"}
Sep 15 13:00:50.868358 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:50.868324    2562 patch_prober.go:28] interesting pod/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 13:00:50.868738 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:50.868393    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 13:00:51.084498 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.084474    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:51.171698 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.171675    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zjtv\" (UniqueName: \"kubernetes.io/projected/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-kube-api-access-9zjtv\") pod \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\" (UID: \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\") "
Sep 15 13:00:51.171835 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.171713    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-maas-api-service-ca\") pod \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\" (UID: \"828a9ef9-3d86-4f92-8632-b6377e0fb1b1\") "
Sep 15 13:00:51.172078 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.172053    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "828a9ef9-3d86-4f92-8632-b6377e0fb1b1" (UID: "828a9ef9-3d86-4f92-8632-b6377e0fb1b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:00:51.172273 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.172257    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:51.173868 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.173844    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-kube-api-access-9zjtv" (OuterVolumeSpecName: "kube-api-access-9zjtv") pod "828a9ef9-3d86-4f92-8632-b6377e0fb1b1" (UID: "828a9ef9-3d86-4f92-8632-b6377e0fb1b1"). InnerVolumeSpecName "kube-api-access-9zjtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:51.273038 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.273010    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zjtv\" (UniqueName: \"kubernetes.io/projected/828a9ef9-3d86-4f92-8632-b6377e0fb1b1-kube-api-access-9zjtv\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:51.312815 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.312781    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:51.312937 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.312840    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:51.352862 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.352835    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:51.867882 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.867849    2562 patch_prober.go:28] interesting pod/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 13:00:51.868060 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.867906    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 13:00:51.904558 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.904536    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"
Sep 15 13:00:51.905008 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.904580    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8" event={"ID":"828a9ef9-3d86-4f92-8632-b6377e0fb1b1","Type":"ContainerDied","Data":"bdc42d391302809e1bba57e56f4a992dc4acb4843616150c79fbb7cee53e4e9f"}
Sep 15 13:00:51.905107 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.905060    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdc42d391302809e1bba57e56f4a992dc4acb4843616150c79fbb7cee53e4e9f"
Sep 15 13:00:51.917126 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:51.917110    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:00:52.871935 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:52.871910    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:52.872871 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:52.872852    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:53.986081 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:53.986022    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"]
Sep 15 13:00:53.988740 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:53.988717    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-179a5f99-99bb2"]
Sep 15 13:00:54.130202 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:54.130172    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"]
Sep 15 13:00:54.130410 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:54.130387    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy" containerID="cri-o://b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99" gracePeriod=30
Sep 15 13:00:54.756561 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:54.756529    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef63b2aa-fdea-45bf-9880-4612bac2cda7" path="/var/lib/kubelet/pods/ef63b2aa-fdea-45bf-9880-4612bac2cda7/volumes"
Sep 15 13:00:59.380435 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.380413    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:59.459765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.459626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-envoy\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.459765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.459702    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c188d78-bfd9-439a-ae26-73858012826e-istio-podinfo\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.459765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.459760    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-socket\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.460086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.459814    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-certs\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.460086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.459860    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-istio-token\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.460086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.459936    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq2q4\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-kube-api-access-mq2q4\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.460086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.459961    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-credential-socket\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.460086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.459999    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c188d78-bfd9-439a-ae26-73858012826e-istiod-ca-cert\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.460086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.460023    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-data\") pod \"8c188d78-bfd9-439a-ae26-73858012826e\" (UID: \"8c188d78-bfd9-439a-ae26-73858012826e\") "
Sep 15 13:00:59.460764 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.460698    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:59.460764 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.460713    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:59.460942 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.460904    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:59.463518 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.462657    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c188d78-bfd9-439a-ae26-73858012826e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:00:59.464227 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.464199    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:59.468095 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.468071    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:59.469206 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.469174    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c188d78-bfd9-439a-ae26-73858012826e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:00:59.469647 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.469617    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-kube-api-access-mq2q4" (OuterVolumeSpecName: "kube-api-access-mq2q4") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "kube-api-access-mq2q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:00:59.470577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.470557    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c188d78-bfd9-439a-ae26-73858012826e" (UID: "8c188d78-bfd9-439a-ae26-73858012826e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:00:59.561513 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561484    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.561513 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561509    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c188d78-bfd9-439a-ae26-73858012826e-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.561670 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561519    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.561670 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561527    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.561670 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561536    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.561670 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561544    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq2q4\" (UniqueName: \"kubernetes.io/projected/8c188d78-bfd9-439a-ae26-73858012826e-kube-api-access-mq2q4\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.561670 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561553    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.561670 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561560    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c188d78-bfd9-439a-ae26-73858012826e-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.561670 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.561568    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c188d78-bfd9-439a-ae26-73858012826e-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:00:59.956471 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.956440    2562 generic.go:356] "Generic (PLEG): container finished" podID="8c188d78-bfd9-439a-ae26-73858012826e" containerID="b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99" exitCode=0
Sep 15 13:00:59.956654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.956495    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"
Sep 15 13:00:59.956654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.956526    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" event={"ID":"8c188d78-bfd9-439a-ae26-73858012826e","Type":"ContainerDied","Data":"b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99"}
Sep 15 13:00:59.956654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.956585    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7" event={"ID":"8c188d78-bfd9-439a-ae26-73858012826e","Type":"ContainerDied","Data":"4db931955cbdf0c43d691f7cc6c5da3a135f2d9a107be959cd62892b2b5fd534"}
Sep 15 13:00:59.956654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.956609    2562 scope.go:117] "RemoveContainer" containerID="b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99"
Sep 15 13:00:59.966331 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.966313    2562 scope.go:117] "RemoveContainer" containerID="7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342"
Sep 15 13:00:59.975534 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.975514    2562 scope.go:117] "RemoveContainer" containerID="b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99"
Sep 15 13:00:59.975833 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:00:59.975813    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99\": container with ID starting with b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99 not found: ID does not exist" containerID="b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99"
Sep 15 13:00:59.975920 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.975840    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99"} err="failed to get container status \"b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99\": rpc error: code = NotFound desc = could not find container \"b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99\": container with ID starting with b0a9bcbff9db41c347691082b9e47edb92a4ee74fa5e686fd22a624276991f99 not found: ID does not exist"
Sep 15 13:00:59.975920 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.975857    2562 scope.go:117] "RemoveContainer" containerID="7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342"
Sep 15 13:00:59.976442 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:00:59.976413    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342\": container with ID starting with 7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342 not found: ID does not exist" containerID="7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342"
Sep 15 13:00:59.976534 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.976451    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342"} err="failed to get container status \"7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342\": rpc error: code = NotFound desc = could not find container \"7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342\": container with ID starting with 7f8f33880d400039666c2cb2e88767d7ddb21712a70c7261fdf6f811461f3342 not found: ID does not exist"
Sep 15 13:00:59.981704 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.981678    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"]
Sep 15 13:00:59.986867 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:00:59.986843    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-179a5f99-openshift-default-7dd95cbc76-v5sp7"]
Sep 15 13:01:00.037254 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:00.037222    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"]
Sep 15 13:01:00.041467 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:00.041438    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-59f4a3-29824620-5c7v6"]
Sep 15 13:01:00.269437 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:00.269343    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"]
Sep 15 13:01:00.272797 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:00.272772    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-59f4a3-dx9k8"]
Sep 15 13:01:00.756699 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:00.756658    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16abf046-395a-4e20-9bd3-74be0b9732c0" path="/var/lib/kubelet/pods/16abf046-395a-4e20-9bd3-74be0b9732c0/volumes"
Sep 15 13:01:00.757100 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:00.757077    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="828a9ef9-3d86-4f92-8632-b6377e0fb1b1" path="/var/lib/kubelet/pods/828a9ef9-3d86-4f92-8632-b6377e0fb1b1/volumes"
Sep 15 13:01:00.757468 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:00.757454    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c188d78-bfd9-439a-ae26-73858012826e" path="/var/lib/kubelet/pods/8c188d78-bfd9-439a-ae26-73858012826e/volumes"
Sep 15 13:01:02.097779 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:02.097742    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:01:17.014086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:17.014048    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:01:29.385747 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:29.385711    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:01:43.243373 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:43.243339    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"]
Sep 15 13:01:43.243866 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:43.243530    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" containerID="cri-o://3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d" gracePeriod=30
Sep 15 13:01:44.002623 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.002585    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"]
Sep 15 13:01:44.003323 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.003298    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy"
Sep 15 13:01:44.003323 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.003321    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy"
Sep 15 13:01:44.003536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.003351    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828a9ef9-3d86-4f92-8632-b6377e0fb1b1" containerName="revoke-keys"
Sep 15 13:01:44.003536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.003360    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="828a9ef9-3d86-4f92-8632-b6377e0fb1b1" containerName="revoke-keys"
Sep 15 13:01:44.003536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.003379    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy"
Sep 15 13:01:44.003536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.003388    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy"
Sep 15 13:01:44.003536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.003500    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy"
Sep 15 13:01:44.003536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.003513    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="828a9ef9-3d86-4f92-8632-b6377e0fb1b1" containerName="revoke-keys"
Sep 15 13:01:44.008329 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.008306    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.012563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.012541    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-413e9f-openshift-default-dockercfg-lwr9n\""
Sep 15 13:01:44.024995 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.024943    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"]
Sep 15 13:01:44.038359 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b45bc3-a44e-4822-93cd-9664eea654ee-istiod-ca-cert\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.038511 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038453    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-token\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.038580 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038543    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-credential-socket\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.038641 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-podinfo\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.038641 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-socket\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.038746 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038656    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-certs\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.038746 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljmdh\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-kube-api-access-ljmdh\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.038859 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-envoy\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.038859 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.038803    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-data\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139558    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-token\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-credential-socket\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-podinfo\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-socket\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-certs\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljmdh\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-kube-api-access-ljmdh\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-envoy\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-data\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.140062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.139990    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b45bc3-a44e-4822-93cd-9664eea654ee-istiod-ca-cert\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.153225 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.140789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b45bc3-a44e-4822-93cd-9664eea654ee-istiod-ca-cert\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.153225 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.141434    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-credential-socket\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.153225 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.142336    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-socket\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.153225 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.142573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-certs\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.153225 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.142792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-data\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.153225 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.145700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-envoy\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.153225 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.152523    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-podinfo\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.159053 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.155727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-token\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.161021 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.160678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljmdh\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-kube-api-access-ljmdh\") pod \"e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") " pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.320087 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.319963    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:44.604548 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.604518    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"]
Sep 15 13:01:44.606170 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:01:44.606133    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b45bc3_a44e_4822_93cd_9664eea654ee.slice/crio-8b1f532b6d087ba6e27fb6d4f52452bf463a8250cb11e6f63ef8ca71b756e480 WatchSource:0}: Error finding container 8b1f532b6d087ba6e27fb6d4f52452bf463a8250cb11e6f63ef8ca71b756e480: Status 404 returned error can't find the container with id 8b1f532b6d087ba6e27fb6d4f52452bf463a8250cb11e6f63ef8ca71b756e480
Sep 15 13:01:44.608589 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.608553    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:01:44.608671 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.608619    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:01:44.608671 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:44.608649    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:01:45.224898 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:45.224859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" event={"ID":"e6b45bc3-a44e-4822-93cd-9664eea654ee","Type":"ContainerStarted","Data":"d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d"}
Sep 15 13:01:45.225068 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:45.224917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" event={"ID":"e6b45bc3-a44e-4822-93cd-9664eea654ee","Type":"ContainerStarted","Data":"8b1f532b6d087ba6e27fb6d4f52452bf463a8250cb11e6f63ef8ca71b756e480"}
Sep 15 13:01:45.251541 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:45.251493    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" podStartSLOduration=2.251475 podStartE2EDuration="2.251475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:01:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:01:45.249838585 +0000 UTC m=+1023.042540321" watchObservedRunningTime="2026-09-15 13:01:45.251475 +0000 UTC m=+1023.044176731"
Sep 15 13:01:45.320933 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:45.320900    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:45.326288 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:45.326262    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:46.231479 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:46.231447    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:46.232849 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:46.232828    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:01:47.098063 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:47.098001    2562 patch_prober.go:28] interesting pod/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 13:01:47.098440 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:47.098088    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 13:01:48.494018 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.493996    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 13:01:48.586973 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.586891    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gxml\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-kube-api-access-6gxml\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.586973 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.586939    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c77919fd-12dc-4604-9e31-f62777722f3b-istio-podinfo\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.587235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c77919fd-12dc-4604-9e31-f62777722f3b-istiod-ca-cert\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.587235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587050    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-data\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.587235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587075    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-certs\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.587235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587130    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-istio-token\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.587235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587176    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-envoy\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.587235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587205    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-credential-socket\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.587235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587231    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-socket\") pod \"c77919fd-12dc-4604-9e31-f62777722f3b\" (UID: \"c77919fd-12dc-4604-9e31-f62777722f3b\") "
Sep 15 13:01:48.587602 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587376    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-data" (OuterVolumeSpecName: "istio-data") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:01:48.587602 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587409    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:01:48.587602 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587423    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c77919fd-12dc-4604-9e31-f62777722f3b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:01:48.587765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587603    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:01:48.587765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587621    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:01:48.587765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587731    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c77919fd-12dc-4604-9e31-f62777722f3b-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:48.587765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587747    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:48.587765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587760    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:48.588051 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587772    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:48.588051 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.587786    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:48.589200 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.589172    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c77919fd-12dc-4604-9e31-f62777722f3b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:01:48.589360 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.589337    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-istio-token" (OuterVolumeSpecName: "istio-token") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:01:48.589425 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.589354    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-kube-api-access-6gxml" (OuterVolumeSpecName: "kube-api-access-6gxml") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "kube-api-access-6gxml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:01:48.589473 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.589420    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c77919fd-12dc-4604-9e31-f62777722f3b" (UID: "c77919fd-12dc-4604-9e31-f62777722f3b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:01:48.689080 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.689053    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c77919fd-12dc-4604-9e31-f62777722f3b-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:48.689080 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.689077    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:48.689249 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.689089    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c77919fd-12dc-4604-9e31-f62777722f3b-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:48.689249 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:48.689101    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gxml\" (UniqueName: \"kubernetes.io/projected/c77919fd-12dc-4604-9e31-f62777722f3b-kube-api-access-6gxml\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:01:49.251667 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.251626    2562 generic.go:356] "Generic (PLEG): container finished" podID="c77919fd-12dc-4604-9e31-f62777722f3b" containerID="3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d" exitCode=0
Sep 15 13:01:49.251861 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.251694    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"
Sep 15 13:01:49.251861 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.251712    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" event={"ID":"c77919fd-12dc-4604-9e31-f62777722f3b","Type":"ContainerDied","Data":"3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d"}
Sep 15 13:01:49.251861 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.251781    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq" event={"ID":"c77919fd-12dc-4604-9e31-f62777722f3b","Type":"ContainerDied","Data":"91fabf6480803fd478c500929dc11d441f15f9d527cec855725fbe7cee21603a"}
Sep 15 13:01:49.251861 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.251811    2562 scope.go:117] "RemoveContainer" containerID="3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d"
Sep 15 13:01:49.261070 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.261055    2562 scope.go:117] "RemoveContainer" containerID="5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350"
Sep 15 13:01:49.269483 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.269467    2562 scope.go:117] "RemoveContainer" containerID="3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d"
Sep 15 13:01:49.269743 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:01:49.269725    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d\": container with ID starting with 3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d not found: ID does not exist" containerID="3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d"
Sep 15 13:01:49.269789 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.269749    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d"} err="failed to get container status \"3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d\": rpc error: code = NotFound desc = could not find container \"3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d\": container with ID starting with 3e36a5f118b947d0b2d22f33fbd022a09f1710183987941a37ee09f66983982d not found: ID does not exist"
Sep 15 13:01:49.269789 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.269764    2562 scope.go:117] "RemoveContainer" containerID="5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350"
Sep 15 13:01:49.270018 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:01:49.269999    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350\": container with ID starting with 5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350 not found: ID does not exist" containerID="5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350"
Sep 15 13:01:49.270093 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.270050    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350"} err="failed to get container status \"5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350\": rpc error: code = NotFound desc = could not find container \"5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350\": container with ID starting with 5c270c09bb8d8f32a1c06f2720656cfd8bb8668d32899307a4859cf792998350 not found: ID does not exist"
Sep 15 13:01:49.279358 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.279337    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"]
Sep 15 13:01:49.286340 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:49.286321    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-59f4a3-openshift-default-7b46c898-kxwpq"]
Sep 15 13:01:50.756072 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:01:50.756020    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" path="/var/lib/kubelet/pods/c77919fd-12dc-4604-9e31-f62777722f3b/volumes"
Sep 15 13:02:00.489001 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:00.488960    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:02:10.392528 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:10.392431    2562 generic.go:356] "Generic (PLEG): container finished" podID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerID="d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d" exitCode=0
Sep 15 13:02:10.392528 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:10.392503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" event={"ID":"e6b45bc3-a44e-4822-93cd-9664eea654ee","Type":"ContainerDied","Data":"d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d"}
Sep 15 13:02:10.392965 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:10.392949    2562 scope.go:117] "RemoveContainer" containerID="d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d"
Sep 15 13:02:10.394132 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:10.394105    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:02:10.394198 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:10.394169    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:02:10.394235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:10.394202    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:02:11.400775 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:11.400743    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" event={"ID":"e6b45bc3-a44e-4822-93cd-9664eea654ee","Type":"ContainerStarted","Data":"a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5"}
Sep 15 13:02:12.320533 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:12.320494    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:02:12.322079 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:12.322020    2562 patch_prober.go:28] interesting pod/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 15 13:02:12.323666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:12.322296    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 15 13:02:13.320525 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:13.320486    2562 patch_prober.go:28] interesting pod/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 15 13:02:13.321021 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:13.320552    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 15 13:02:14.324517 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:14.324484    2562 patch_prober.go:28] interesting pod/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 15 13:02:14.325574 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:14.325543    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 15 13:02:15.321286 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:15.321247    2562 patch_prober.go:28] interesting pod/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 15 13:02:15.321585 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:15.321553    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 15 13:02:16.231599 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:16.231564    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:02:16.320930 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:16.320886    2562 patch_prober.go:28] interesting pod/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 15 13:02:16.321165 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:16.320963    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 15 13:02:17.325955 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:17.325919    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:02:17.327163 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:17.327137    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:02:24.586768 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:24.586733    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:02:38.370828 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.370738    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"]
Sep 15 13:02:38.371341 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.371322    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy"
Sep 15 13:02:38.371341 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.371342    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy"
Sep 15 13:02:38.371484 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.371414    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy"
Sep 15 13:02:38.371484 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.371427    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy"
Sep 15 13:02:38.371484 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.371433    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c188d78-bfd9-439a-ae26-73858012826e" containerName="istio-proxy"
Sep 15 13:02:38.374321 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.374292    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.378139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.378116    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-6c9f0b\"/\"kube-root-ca.crt\""
Sep 15 13:02:38.378305 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.378149    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-6c9f0b\"/\"openshift-service-ca.crt\""
Sep 15 13:02:38.378305 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.378170    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-6c9f0b\"/\"premium-w0-6c9f0b-kserve-self-signed-certs\""
Sep 15 13:02:38.378479 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.378171    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-6c9f0b\"/\"default-dockercfg-g2hgd\""
Sep 15 13:02:38.383308 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.383282    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"]
Sep 15 13:02:38.530591 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.530555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsn4f\" (UniqueName: \"kubernetes.io/projected/212d55d6-91d9-456d-ae90-e6d0a61a08bb-kube-api-access-nsn4f\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.530773 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.530605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-dshm\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.530773 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.530709    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tls-certs\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.530870 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.530790    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tmp-dir\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.530870 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.530838    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-model-cache\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.530951 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.530874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-home\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.631552 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.631465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-model-cache\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.631552 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.631520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-home\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.631796 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.631570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsn4f\" (UniqueName: \"kubernetes.io/projected/212d55d6-91d9-456d-ae90-e6d0a61a08bb-kube-api-access-nsn4f\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.631796 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.631604    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-dshm\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.631796 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.631672    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tls-certs\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.631796 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.631735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tmp-dir\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.631989 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.631918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-model-cache\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.631989 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.631941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-home\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.632152 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.632128    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tmp-dir\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.633867 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.633844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-dshm\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.634130 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.634114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tls-certs\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.642072 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.642050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsn4f\" (UniqueName: \"kubernetes.io/projected/212d55d6-91d9-456d-ae90-e6d0a61a08bb-kube-api-access-nsn4f\") pod \"premium-w0-6c9f0b-kserve-9d545964b-l8ptx\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") " pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.685013 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.684986    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:38.964510 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:02:38.964479    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod212d55d6_91d9_456d_ae90_e6d0a61a08bb.slice/crio-45ed707ebdfc9dafd4991ee0de3a6caf788c972fa5d69912243b0cc0a980e744 WatchSource:0}: Error finding container 45ed707ebdfc9dafd4991ee0de3a6caf788c972fa5d69912243b0cc0a980e744: Status 404 returned error can't find the container with id 45ed707ebdfc9dafd4991ee0de3a6caf788c972fa5d69912243b0cc0a980e744
Sep 15 13:02:38.965086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:38.965059    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"]
Sep 15 13:02:39.578370 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:39.578336    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx" event={"ID":"212d55d6-91d9-456d-ae90-e6d0a61a08bb","Type":"ContainerStarted","Data":"e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5"}
Sep 15 13:02:39.578858 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:39.578381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx" event={"ID":"212d55d6-91d9-456d-ae90-e6d0a61a08bb","Type":"ContainerStarted","Data":"45ed707ebdfc9dafd4991ee0de3a6caf788c972fa5d69912243b0cc0a980e744"}
Sep 15 13:02:39.606197 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:39.606097    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx" podStartSLOduration=1.606079832 podStartE2EDuration="1.606079832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:02:39.601075773 +0000 UTC m=+1077.393777503" watchObservedRunningTime="2026-09-15 13:02:39.606079832 +0000 UTC m=+1077.398781562"
Sep 15 13:02:46.498577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:46.498536    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:02:48.685416 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:48.685377    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:48.685833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:48.685427    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:48.698185 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:48.698152    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:49.649811 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:49.649781    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:02:59.193562 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.193529    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"]
Sep 15 13:02:59.194014 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.193981    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy"
Sep 15 13:02:59.194014 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.193993    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77919fd-12dc-4604-9e31-f62777722f3b" containerName="istio-proxy"
Sep 15 13:02:59.196967 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.196944    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.217376 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:02:59.217346    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-1c35dd\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="0152c575-0a44-474c-9d4a-eda58f4aedb1" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.217497 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:02:59.217422    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"test-model-1c35dd-kserve-self-signed-certs\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-1c35dd\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-1c35dd\"/\"test-model-1c35dd-kserve-self-signed-certs\"" type="*v1.Secret"
Sep 15 13:02:59.217497 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:02:59.217471    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-1c35dd\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-1c35dd\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 15 13:02:59.217497 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:02:59.217472    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-6hhc4\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-1c35dd\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-1c35dd\"/\"default-dockercfg-6hhc4\"" type="*v1.Secret"
Sep 15 13:02:59.217729 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:02:59.217709    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-1c35dd\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-1c35dd\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 15 13:02:59.241834 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.241798    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"]
Sep 15 13:02:59.357441 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.357409    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-tmp-dir\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.357628 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.357484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0152c575-0a44-474c-9d4a-eda58f4aedb1-tls-certs\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.357628 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.357514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xslq9\" (UniqueName: \"kubernetes.io/projected/0152c575-0a44-474c-9d4a-eda58f4aedb1-kube-api-access-xslq9\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.357628 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.357584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-model-cache\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.357760 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.357652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-home\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.357760 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.357669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-dshm\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.459796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-model-cache\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.459898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-home\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.459929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-dshm\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.460013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-tmp-dir\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.460152    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0152c575-0a44-474c-9d4a-eda58f4aedb1-tls-certs\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.460195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xslq9\" (UniqueName: \"kubernetes.io/projected/0152c575-0a44-474c-9d4a-eda58f4aedb1-kube-api-access-xslq9\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.460984    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-model-cache\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.461275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-home\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.464012    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-dshm\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:02:59.466501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:02:59.464294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-tmp-dir\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:03:00.053909 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:00.053872    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1c35dd\"/\"test-model-1c35dd-kserve-self-signed-certs\""
Sep 15 13:03:00.057419 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:00.057389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0152c575-0a44-474c-9d4a-eda58f4aedb1-tls-certs\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:03:00.098632 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:00.098598    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1c35dd\"/\"openshift-service-ca.crt\""
Sep 15 13:03:00.469154 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:00.469122    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1c35dd\"/\"kube-root-ca.crt\""
Sep 15 13:03:00.480570 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:00.480543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xslq9\" (UniqueName: \"kubernetes.io/projected/0152c575-0a44-474c-9d4a-eda58f4aedb1-kube-api-access-xslq9\") pod \"test-model-1c35dd-kserve-8fc8d4849-dsbsk\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") " pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:03:00.662926 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:00.662889    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1c35dd\"/\"default-dockercfg-6hhc4\""
Sep 15 13:03:00.706146 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:00.706118    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:03:00.998565 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:00.995724    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"]
Sep 15 13:03:01.712946 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:01.712907    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk" event={"ID":"0152c575-0a44-474c-9d4a-eda58f4aedb1","Type":"ContainerStarted","Data":"995b478b8d040a6441bb2abba9f7ab71354e5b192564075a3a70b54a33fbe658"}
Sep 15 13:03:01.712946 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:01.712951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk" event={"ID":"0152c575-0a44-474c-9d4a-eda58f4aedb1","Type":"ContainerStarted","Data":"303dd30a5f3b7155315b67a2743bbff10d421747cd11c020aa3c1d356ff4612b"}
Sep 15 13:03:01.750334 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:01.750281    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk" podStartSLOduration=2.750262818 podStartE2EDuration="2.750262818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:02:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:03:01.747671787 +0000 UTC m=+1099.540373528" watchObservedRunningTime="2026-09-15 13:03:01.750262818 +0000 UTC m=+1099.542964550"
Sep 15 13:03:10.137079 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:10.137006    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:03:10.706577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:10.706541    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:03:10.706749 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:10.706603    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:03:10.719510 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:10.719488    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:03:10.783141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:10.783111    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:03:38.290458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.290425    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"]
Sep 15 13:03:38.294412 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.294389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:38.298520 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.298500    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:03:38.299692 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.299674    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:03:38.299767 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.299680    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:03:38.311695 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.311672    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"]
Sep 15 13:03:38.373600 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.373572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05df5cbe-fd35-4e35-9e53-6965397689f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c\" (UID: \"05df5cbe-fd35-4e35-9e53-6965397689f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:38.373600 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.373602    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st6fm\" (UniqueName: \"kubernetes.io/projected/05df5cbe-fd35-4e35-9e53-6965397689f8-kube-api-access-st6fm\") pod \"maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c\" (UID: \"05df5cbe-fd35-4e35-9e53-6965397689f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:38.474913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.474879    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05df5cbe-fd35-4e35-9e53-6965397689f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c\" (UID: \"05df5cbe-fd35-4e35-9e53-6965397689f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:38.474913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.474915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st6fm\" (UniqueName: \"kubernetes.io/projected/05df5cbe-fd35-4e35-9e53-6965397689f8-kube-api-access-st6fm\") pod \"maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c\" (UID: \"05df5cbe-fd35-4e35-9e53-6965397689f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:38.475595 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.475572    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05df5cbe-fd35-4e35-9e53-6965397689f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c\" (UID: \"05df5cbe-fd35-4e35-9e53-6965397689f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:38.486198 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.486173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st6fm\" (UniqueName: \"kubernetes.io/projected/05df5cbe-fd35-4e35-9e53-6965397689f8-kube-api-access-st6fm\") pod \"maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c\" (UID: \"05df5cbe-fd35-4e35-9e53-6965397689f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:38.605274 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.605201    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:38.887739 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.887709    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"]
Sep 15 13:03:38.889094 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:03:38.889062    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05df5cbe_fd35_4e35_9e53_6965397689f8.slice/crio-fb16edf675d7b27283df7b92f9affea902dd0e9016c928fd793e619a711033e1 WatchSource:0}: Error finding container fb16edf675d7b27283df7b92f9affea902dd0e9016c928fd793e619a711033e1: Status 404 returned error can't find the container with id fb16edf675d7b27283df7b92f9affea902dd0e9016c928fd793e619a711033e1
Sep 15 13:03:38.942283 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:38.942254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c" event={"ID":"05df5cbe-fd35-4e35-9e53-6965397689f8","Type":"ContainerStarted","Data":"fb16edf675d7b27283df7b92f9affea902dd0e9016c928fd793e619a711033e1"}
Sep 15 13:03:39.950663 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:39.950628    2562 generic.go:356] "Generic (PLEG): container finished" podID="05df5cbe-fd35-4e35-9e53-6965397689f8" containerID="9650d94926b3ab49700b36e4cc39b7c5b4dca08ffd792d1a5c1e42b02cb14af3" exitCode=0
Sep 15 13:03:39.951109 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:39.950682    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c" event={"ID":"05df5cbe-fd35-4e35-9e53-6965397689f8","Type":"ContainerDied","Data":"9650d94926b3ab49700b36e4cc39b7c5b4dca08ffd792d1a5c1e42b02cb14af3"}
Sep 15 13:03:41.090069 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.090044    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:41.101113 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.101092    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st6fm\" (UniqueName: \"kubernetes.io/projected/05df5cbe-fd35-4e35-9e53-6965397689f8-kube-api-access-st6fm\") pod \"05df5cbe-fd35-4e35-9e53-6965397689f8\" (UID: \"05df5cbe-fd35-4e35-9e53-6965397689f8\") "
Sep 15 13:03:41.101233 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.101221    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05df5cbe-fd35-4e35-9e53-6965397689f8-maas-api-service-ca\") pod \"05df5cbe-fd35-4e35-9e53-6965397689f8\" (UID: \"05df5cbe-fd35-4e35-9e53-6965397689f8\") "
Sep 15 13:03:41.101589 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.101565    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05df5cbe-fd35-4e35-9e53-6965397689f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05df5cbe-fd35-4e35-9e53-6965397689f8" (UID: "05df5cbe-fd35-4e35-9e53-6965397689f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:03:41.103105 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.103075    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05df5cbe-fd35-4e35-9e53-6965397689f8-kube-api-access-st6fm" (OuterVolumeSpecName: "kube-api-access-st6fm") pod "05df5cbe-fd35-4e35-9e53-6965397689f8" (UID: "05df5cbe-fd35-4e35-9e53-6965397689f8"). InnerVolumeSpecName "kube-api-access-st6fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:03:41.202806 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.202775    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05df5cbe-fd35-4e35-9e53-6965397689f8-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:03:41.202806 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.202802    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st6fm\" (UniqueName: \"kubernetes.io/projected/05df5cbe-fd35-4e35-9e53-6965397689f8-kube-api-access-st6fm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:03:41.222496 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.222403    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:03:41.965260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.965230    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"
Sep 15 13:03:41.965461 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.965267    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c" event={"ID":"05df5cbe-fd35-4e35-9e53-6965397689f8","Type":"ContainerDied","Data":"fb16edf675d7b27283df7b92f9affea902dd0e9016c928fd793e619a711033e1"}
Sep 15 13:03:41.965461 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:41.965341    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb16edf675d7b27283df7b92f9affea902dd0e9016c928fd793e619a711033e1"
Sep 15 13:03:49.564120 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:49.564083    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"]
Sep 15 13:03:49.566369 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:49.566346    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-413e9f-m4p8c"]
Sep 15 13:03:50.756448 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:50.756413    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05df5cbe-fd35-4e35-9e53-6965397689f8" path="/var/lib/kubelet/pods/05df5cbe-fd35-4e35-9e53-6965397689f8/volumes"
Sep 15 13:03:58.383700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:58.383658    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"]
Sep 15 13:03:58.384203 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:58.383914    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy" containerID="cri-o://a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5" gracePeriod=30
Sep 15 13:03:59.334628 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.334599    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"]
Sep 15 13:03:59.335060 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.335044    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05df5cbe-fd35-4e35-9e53-6965397689f8" containerName="revoke-keys"
Sep 15 13:03:59.335124 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.335062    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="05df5cbe-fd35-4e35-9e53-6965397689f8" containerName="revoke-keys"
Sep 15 13:03:59.335168 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.335159    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="05df5cbe-fd35-4e35-9e53-6965397689f8" containerName="revoke-keys"
Sep 15 13:03:59.339790 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.339768    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.365270 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.365253    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-08b44c-openshift-default-dockercfg-nlq44\""
Sep 15 13:03:59.404701 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.404675    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"]
Sep 15 13:03:59.508159 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508126    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-certs\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.508326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38f46407-d42c-410f-aa2e-184049f15499-istiod-ca-cert\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.508326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-credential-socket\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.508326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-envoy\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.508326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508323    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-socket\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.508542 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-data\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.508542 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-istio-token\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.508542 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r67hv\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-kube-api-access-r67hv\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.508663 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.508557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38f46407-d42c-410f-aa2e-184049f15499-istio-podinfo\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609527 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-certs\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609527 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38f46407-d42c-410f-aa2e-184049f15499-istiod-ca-cert\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609527 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-credential-socket\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609807 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-envoy\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609807 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609568    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-socket\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609807 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609594    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-data\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609807 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609661    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-istio-token\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609807 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67hv\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-kube-api-access-r67hv\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.609807 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38f46407-d42c-410f-aa2e-184049f15499-istio-podinfo\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.610134 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-certs\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.610134 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609910    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-credential-socket\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.610134 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.609991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-socket\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.610293 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.610202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-data\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.610351 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.610326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38f46407-d42c-410f-aa2e-184049f15499-istiod-ca-cert\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.612050 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.612010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-envoy\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.612419 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.612402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38f46407-d42c-410f-aa2e-184049f15499-istio-podinfo\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.623447 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.623421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r67hv\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-kube-api-access-r67hv\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.634573 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.634546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-istio-token\") pod \"e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") " pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.650397 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.650367    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:03:59.967075 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.967050    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"]
Sep 15 13:03:59.969155 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:03:59.969122    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f46407_d42c_410f_aa2e_184049f15499.slice/crio-f8ef9b3460f94b309c3e32deb10e73602e1d198978f550a54f04bc2dba6d078e WatchSource:0}: Error finding container f8ef9b3460f94b309c3e32deb10e73602e1d198978f550a54f04bc2dba6d078e: Status 404 returned error can't find the container with id f8ef9b3460f94b309c3e32deb10e73602e1d198978f550a54f04bc2dba6d078e
Sep 15 13:03:59.971367 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.971334    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:03:59.971431 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.971401    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:03:59.971475 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:03:59.971428    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:04:00.085255 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:00.085214    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf" event={"ID":"38f46407-d42c-410f-aa2e-184049f15499","Type":"ContainerStarted","Data":"72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba"}
Sep 15 13:04:00.085436 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:00.085288    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf" event={"ID":"38f46407-d42c-410f-aa2e-184049f15499","Type":"ContainerStarted","Data":"f8ef9b3460f94b309c3e32deb10e73602e1d198978f550a54f04bc2dba6d078e"}
Sep 15 13:04:00.127321 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:00.127257    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf" podStartSLOduration=1.127232357 podStartE2EDuration="1.127232357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:04:00.12659442 +0000 UTC m=+1157.919296155" watchObservedRunningTime="2026-09-15 13:04:00.127232357 +0000 UTC m=+1157.919934090"
Sep 15 13:04:00.651640 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:00.651601    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:04:01.657385 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:01.657356    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:04:02.097474 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:02.097442    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:04:02.098361 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:02.098338    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:04:02.329100 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:02.329068    2562 patch_prober.go:28] interesting pod/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 13:04:02.329258 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:02.329128    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 13:04:03.647994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.647934    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:04:03.753768 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.753737    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-certs\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.753940 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.753805    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-data\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.753940 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.753853    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljmdh\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-kube-api-access-ljmdh\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.753940 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.753876    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b45bc3-a44e-4822-93cd-9664eea654ee-istiod-ca-cert\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.753940 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.753901    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-envoy\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.754250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.753977    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-socket\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.754250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.753999    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:03.754250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754057    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-token\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.754250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754081    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-podinfo\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.754250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754117    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-credential-socket\") pod \"e6b45bc3-a44e-4822-93cd-9664eea654ee\" (UID: \"e6b45bc3-a44e-4822-93cd-9664eea654ee\") "
Sep 15 13:04:03.754250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754166    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:03.754543 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754367    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b45bc3-a44e-4822-93cd-9664eea654ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:04:03.754543 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754442    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:03.754644 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754588    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:03.754644 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754610    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:03.754644 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754624    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:03.754644 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754588    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:03.754906 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.754664    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b45bc3-a44e-4822-93cd-9664eea654ee-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:03.756190 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.756167    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:04:03.756293 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.756231    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-kube-api-access-ljmdh" (OuterVolumeSpecName: "kube-api-access-ljmdh") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "kube-api-access-ljmdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:04:03.756338 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.756304    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:04:03.756404 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.756387    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e6b45bc3-a44e-4822-93cd-9664eea654ee" (UID: "e6b45bc3-a44e-4822-93cd-9664eea654ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:03.855928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.855896    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljmdh\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-kube-api-access-ljmdh\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:03.855928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.855921    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:03.855928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.855931    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b45bc3-a44e-4822-93cd-9664eea654ee-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:03.856180 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.855939    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:03.856180 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:03.855948    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b45bc3-a44e-4822-93cd-9664eea654ee-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:04.111220 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.111187    2562 generic.go:356] "Generic (PLEG): container finished" podID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerID="a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5" exitCode=0
Sep 15 13:04:04.111400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.111233    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"
Sep 15 13:04:04.111400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.111272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" event={"ID":"e6b45bc3-a44e-4822-93cd-9664eea654ee","Type":"ContainerDied","Data":"a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5"}
Sep 15 13:04:04.111400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.111332    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k" event={"ID":"e6b45bc3-a44e-4822-93cd-9664eea654ee","Type":"ContainerDied","Data":"8b1f532b6d087ba6e27fb6d4f52452bf463a8250cb11e6f63ef8ca71b756e480"}
Sep 15 13:04:04.111400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.111356    2562 scope.go:117] "RemoveContainer" containerID="a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5"
Sep 15 13:04:04.122111 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.122084    2562 scope.go:117] "RemoveContainer" containerID="d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d"
Sep 15 13:04:04.132230 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.132210    2562 scope.go:117] "RemoveContainer" containerID="a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5"
Sep 15 13:04:04.132528 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:04:04.132509    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5\": container with ID starting with a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5 not found: ID does not exist" containerID="a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5"
Sep 15 13:04:04.132580 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.132535    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5"} err="failed to get container status \"a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5\": rpc error: code = NotFound desc = could not find container \"a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5\": container with ID starting with a3c717764b75c19405b6f8968142faee92c2035e43c23d4d956e6cdf1b334fe5 not found: ID does not exist"
Sep 15 13:04:04.132580 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.132552    2562 scope.go:117] "RemoveContainer" containerID="d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d"
Sep 15 13:04:04.132785 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:04:04.132766    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d\": container with ID starting with d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d not found: ID does not exist" containerID="d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d"
Sep 15 13:04:04.132841 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.132795    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d"} err="failed to get container status \"d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d\": rpc error: code = NotFound desc = could not find container \"d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d\": container with ID starting with d1e7c6e8abaddf20e260d091218381c34bf4a2aa4da89650162cbf607403a28d not found: ID does not exist"
Sep 15 13:04:04.166560 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.166537    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"]
Sep 15 13:04:04.172023 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.172004    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-413e9f-openshift-default-7b67c47764-qfq5k"]
Sep 15 13:04:04.756952 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:04.756918    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" path="/var/lib/kubelet/pods/e6b45bc3-a44e-4822-93cd-9664eea654ee/volumes"
Sep 15 13:04:08.439910 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:08.439874    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:04:29.203646 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:29.203609    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"]
Sep 15 13:04:29.204229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:29.203834    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf" podUID="38f46407-d42c-410f-aa2e-184049f15499" containerName="istio-proxy" containerID="cri-o://72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba" gracePeriod=30
Sep 15 13:04:32.100083 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:32.100015    2562 patch_prober.go:28] interesting pod/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 13:04:32.100453 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:32.100114    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf" podUID="38f46407-d42c-410f-aa2e-184049f15499" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 13:04:32.416799 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:32.416714    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:04:34.468097 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.467996    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:04:34.584113 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584069    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-data\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584113 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584128    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r67hv\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-kube-api-access-r67hv\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584262    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38f46407-d42c-410f-aa2e-184049f15499-istiod-ca-cert\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584321    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-socket\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584385    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-data" (OuterVolumeSpecName: "istio-data") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:34.584558 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584439    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-certs\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584558 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584502    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-credential-socket\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584558 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584531    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:34.584684 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584565    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-istio-token\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584684 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584609    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38f46407-d42c-410f-aa2e-184049f15499-istio-podinfo\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584684 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584605    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:34.584684 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584640    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-envoy\") pod \"38f46407-d42c-410f-aa2e-184049f15499\" (UID: \"38f46407-d42c-410f-aa2e-184049f15499\") "
Sep 15 13:04:34.584909 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584622    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38f46407-d42c-410f-aa2e-184049f15499-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:04:34.584909 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.584716    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:34.585167 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.585146    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:34.585260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.585173    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38f46407-d42c-410f-aa2e-184049f15499-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:34.585260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.585189    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:34.585260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.585203    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:34.585260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.585218    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:34.586601 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.586570    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:04:34.586700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.586638    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-istio-token" (OuterVolumeSpecName: "istio-token") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:04:34.586700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.586652    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38f46407-d42c-410f-aa2e-184049f15499-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:04:34.586700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.586651    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-kube-api-access-r67hv" (OuterVolumeSpecName: "kube-api-access-r67hv") pod "38f46407-d42c-410f-aa2e-184049f15499" (UID: "38f46407-d42c-410f-aa2e-184049f15499"). InnerVolumeSpecName "kube-api-access-r67hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:04:34.685982 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.685947    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:34.685982 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.685975    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38f46407-d42c-410f-aa2e-184049f15499-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:34.685982 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.685983    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38f46407-d42c-410f-aa2e-184049f15499-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:34.685982 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:34.685993    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r67hv\" (UniqueName: \"kubernetes.io/projected/38f46407-d42c-410f-aa2e-184049f15499-kube-api-access-r67hv\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:04:35.297179 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.297144    2562 generic.go:356] "Generic (PLEG): container finished" podID="38f46407-d42c-410f-aa2e-184049f15499" containerID="72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba" exitCode=0
Sep 15 13:04:35.297393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.297198    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"
Sep 15 13:04:35.297393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.297206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf" event={"ID":"38f46407-d42c-410f-aa2e-184049f15499","Type":"ContainerDied","Data":"72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba"}
Sep 15 13:04:35.297393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.297247    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf" event={"ID":"38f46407-d42c-410f-aa2e-184049f15499","Type":"ContainerDied","Data":"f8ef9b3460f94b309c3e32deb10e73602e1d198978f550a54f04bc2dba6d078e"}
Sep 15 13:04:35.297393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.297268    2562 scope.go:117] "RemoveContainer" containerID="72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba"
Sep 15 13:04:35.306811 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.306792    2562 scope.go:117] "RemoveContainer" containerID="72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba"
Sep 15 13:04:35.307085 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:04:35.307067    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba\": container with ID starting with 72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba not found: ID does not exist" containerID="72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba"
Sep 15 13:04:35.307145 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.307093    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba"} err="failed to get container status \"72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba\": rpc error: code = NotFound desc = could not find container \"72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba\": container with ID starting with 72895f0425ea519959d262a78d61e7cdcefe9df93cd5fe9e606f778fedf178ba not found: ID does not exist"
Sep 15 13:04:35.321153 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.321130    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"]
Sep 15 13:04:35.327668 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:35.327642    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-08b44c-openshift-default-868dfdf674-6x9wf"]
Sep 15 13:04:36.756423 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:36.756387    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38f46407-d42c-410f-aa2e-184049f15499" path="/var/lib/kubelet/pods/38f46407-d42c-410f-aa2e-184049f15499/volumes"
Sep 15 13:04:42.341634 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.341603    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"]
Sep 15 13:04:42.342074 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342060    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38f46407-d42c-410f-aa2e-184049f15499" containerName="istio-proxy"
Sep 15 13:04:42.342074 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342075    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="38f46407-d42c-410f-aa2e-184049f15499" containerName="istio-proxy"
Sep 15 13:04:42.342159 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342087    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy"
Sep 15 13:04:42.342159 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342093    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy"
Sep 15 13:04:42.342159 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342106    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy"
Sep 15 13:04:42.342159 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342111    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy"
Sep 15 13:04:42.342278 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342182    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="38f46407-d42c-410f-aa2e-184049f15499" containerName="istio-proxy"
Sep 15 13:04:42.342278 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342195    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy"
Sep 15 13:04:42.342278 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.342201    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b45bc3-a44e-4822-93cd-9664eea654ee" containerName="istio-proxy"
Sep 15 13:04:42.345297 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.345275    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.350851 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.350814    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-2af9a3\"/\"openshift-service-ca.crt\""
Sep 15 13:04:42.350975 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.350858    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2af9a3\"/\"e2e-conflict-b-10e04108-kserve-self-signed-certs\""
Sep 15 13:04:42.350975 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.350934    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2af9a3\"/\"default-dockercfg-b2lfg\""
Sep 15 13:04:42.352130 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.352107    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-2af9a3\"/\"kube-root-ca.crt\""
Sep 15 13:04:42.363203 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.363179    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"]
Sep 15 13:04:42.465213 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.465180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-dshm\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.465213 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.465223    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3287deb5-7448-43eb-9724-6a3b0142b6df-tls-certs\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.465424 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.465258    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-tmp-dir\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.465424 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.465312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjsd\" (UniqueName: \"kubernetes.io/projected/3287deb5-7448-43eb-9724-6a3b0142b6df-kube-api-access-kwjsd\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.465424 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.465360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-model-cache\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.465424 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.465389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-home\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566046 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjsd\" (UniqueName: \"kubernetes.io/projected/3287deb5-7448-43eb-9724-6a3b0142b6df-kube-api-access-kwjsd\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566214 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-model-cache\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566214 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-home\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566214 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-dshm\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566365 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566221    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3287deb5-7448-43eb-9724-6a3b0142b6df-tls-certs\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566365 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-tmp-dir\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566538 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566515    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-model-cache\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566605 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-home\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.566658 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.566583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-tmp-dir\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.568460 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.568439    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-dshm\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.568800 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.568782    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3287deb5-7448-43eb-9724-6a3b0142b6df-tls-certs\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.579910 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.579891    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjsd\" (UniqueName: \"kubernetes.io/projected/3287deb5-7448-43eb-9724-6a3b0142b6df-kube-api-access-kwjsd\") pod \"e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") " pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.659762 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.659685    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2af9a3\"/\"default-dockercfg-b2lfg\""
Sep 15 13:04:42.666870 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.666847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:42.769329 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.769302    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:04:42.773019 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.772996    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:04:42.774868 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.774850    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:04:42.777237 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.777220    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:04:42.956283 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:42.953252    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"]
Sep 15 13:04:42.957071 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:04:42.957002    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3287deb5_7448_43eb_9724_6a3b0142b6df.slice/crio-bc30bbc0808515a2537e3c9814340acf1aaf71808e659159331b11641a1bfc75 WatchSource:0}: Error finding container bc30bbc0808515a2537e3c9814340acf1aaf71808e659159331b11641a1bfc75: Status 404 returned error can't find the container with id bc30bbc0808515a2537e3c9814340acf1aaf71808e659159331b11641a1bfc75
Sep 15 13:04:43.346612 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:43.346524    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk" event={"ID":"3287deb5-7448-43eb-9724-6a3b0142b6df","Type":"ContainerStarted","Data":"d8aaeecd4723c8a6a126c18d1c88c18c480a90f560b1889533a01d6eb524bcbd"}
Sep 15 13:04:43.346612 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:43.346566    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk" event={"ID":"3287deb5-7448-43eb-9724-6a3b0142b6df","Type":"ContainerStarted","Data":"bc30bbc0808515a2537e3c9814340acf1aaf71808e659159331b11641a1bfc75"}
Sep 15 13:04:43.380940 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:43.380890    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk" podStartSLOduration=1.380872498 podStartE2EDuration="1.380872498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:04:43.378917733 +0000 UTC m=+1201.171619465" watchObservedRunningTime="2026-09-15 13:04:43.380872498 +0000 UTC m=+1201.173574228"
Sep 15 13:04:50.986866 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:50.986833    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:04:52.667632 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:52.667595    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:52.668116 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:52.667651    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:52.680273 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:52.680246    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:53.416102 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:53.416073    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:04:57.369943 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:04:57.369910    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"]
Sep 15 13:05:06.014154 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:06.014118    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"]
Sep 15 13:05:12.391159 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.391094    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk" podUID="3287deb5-7448-43eb-9724-6a3b0142b6df" containerName="main" containerID="cri-o://d8aaeecd4723c8a6a126c18d1c88c18c480a90f560b1889533a01d6eb524bcbd" gracePeriod=45
Sep 15 13:05:12.525605 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.525574    2562 generic.go:356] "Generic (PLEG): container finished" podID="3287deb5-7448-43eb-9724-6a3b0142b6df" containerID="d8aaeecd4723c8a6a126c18d1c88c18c480a90f560b1889533a01d6eb524bcbd" exitCode=0
Sep 15 13:05:12.525781 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.525657    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk" event={"ID":"3287deb5-7448-43eb-9724-6a3b0142b6df","Type":"ContainerDied","Data":"d8aaeecd4723c8a6a126c18d1c88c18c480a90f560b1889533a01d6eb524bcbd"}
Sep 15 13:05:12.653540 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.653483    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:05:12.792972 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.792943    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-home\") pod \"3287deb5-7448-43eb-9724-6a3b0142b6df\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") "
Sep 15 13:05:12.793167 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.792990    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-dshm\") pod \"3287deb5-7448-43eb-9724-6a3b0142b6df\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") "
Sep 15 13:05:12.793167 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793018    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-tmp-dir\") pod \"3287deb5-7448-43eb-9724-6a3b0142b6df\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") "
Sep 15 13:05:12.793167 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793070    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwjsd\" (UniqueName: \"kubernetes.io/projected/3287deb5-7448-43eb-9724-6a3b0142b6df-kube-api-access-kwjsd\") pod \"3287deb5-7448-43eb-9724-6a3b0142b6df\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") "
Sep 15 13:05:12.793337 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793222    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-home" (OuterVolumeSpecName: "home") pod "3287deb5-7448-43eb-9724-6a3b0142b6df" (UID: "3287deb5-7448-43eb-9724-6a3b0142b6df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:12.793337 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793262    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3287deb5-7448-43eb-9724-6a3b0142b6df-tls-certs\") pod \"3287deb5-7448-43eb-9724-6a3b0142b6df\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") "
Sep 15 13:05:12.793337 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793265    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3287deb5-7448-43eb-9724-6a3b0142b6df" (UID: "3287deb5-7448-43eb-9724-6a3b0142b6df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:12.793337 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793324    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-model-cache\") pod \"3287deb5-7448-43eb-9724-6a3b0142b6df\" (UID: \"3287deb5-7448-43eb-9724-6a3b0142b6df\") "
Sep 15 13:05:12.793625 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793602    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-model-cache" (OuterVolumeSpecName: "model-cache") pod "3287deb5-7448-43eb-9724-6a3b0142b6df" (UID: "3287deb5-7448-43eb-9724-6a3b0142b6df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:12.793708 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793686    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-model-cache\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:12.793708 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793710    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-home\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:12.793901 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.793723    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-tmp-dir\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:12.795269 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.795244    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-dshm" (OuterVolumeSpecName: "dshm") pod "3287deb5-7448-43eb-9724-6a3b0142b6df" (UID: "3287deb5-7448-43eb-9724-6a3b0142b6df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:12.795380 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.795322    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3287deb5-7448-43eb-9724-6a3b0142b6df-kube-api-access-kwjsd" (OuterVolumeSpecName: "kube-api-access-kwjsd") pod "3287deb5-7448-43eb-9724-6a3b0142b6df" (UID: "3287deb5-7448-43eb-9724-6a3b0142b6df"). InnerVolumeSpecName "kube-api-access-kwjsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:05:12.795448 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.795389    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3287deb5-7448-43eb-9724-6a3b0142b6df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3287deb5-7448-43eb-9724-6a3b0142b6df" (UID: "3287deb5-7448-43eb-9724-6a3b0142b6df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:05:12.895214 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.895179    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3287deb5-7448-43eb-9724-6a3b0142b6df-dshm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:12.895214 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.895208    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwjsd\" (UniqueName: \"kubernetes.io/projected/3287deb5-7448-43eb-9724-6a3b0142b6df-kube-api-access-kwjsd\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:12.895214 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.895222    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3287deb5-7448-43eb-9724-6a3b0142b6df-tls-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:12.898535 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:12.898509    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:05:13.533798 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:13.533768    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk" event={"ID":"3287deb5-7448-43eb-9724-6a3b0142b6df","Type":"ContainerDied","Data":"bc30bbc0808515a2537e3c9814340acf1aaf71808e659159331b11641a1bfc75"}
Sep 15 13:05:13.533798 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:13.533799    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"
Sep 15 13:05:13.534309 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:13.533820    2562 scope.go:117] "RemoveContainer" containerID="d8aaeecd4723c8a6a126c18d1c88c18c480a90f560b1889533a01d6eb524bcbd"
Sep 15 13:05:13.583389 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:13.583347    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"]
Sep 15 13:05:13.585888 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:13.585849    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2af9a3/e2e-conflict-b-10e04108-kserve-5bb44996b-khgpk"]
Sep 15 13:05:14.757813 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:14.757776    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3287deb5-7448-43eb-9724-6a3b0142b6df" path="/var/lib/kubelet/pods/3287deb5-7448-43eb-9724-6a3b0142b6df/volumes"
Sep 15 13:05:21.037833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.037763    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm" podUID="f6f971ee-83ba-4a5e-b370-58488310ea21" containerName="main" containerID="cri-o://4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538" gracePeriod=45
Sep 15 13:05:21.294431 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.294370    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:05:21.380938 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.380905    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nw6s\" (UniqueName: \"kubernetes.io/projected/f6f971ee-83ba-4a5e-b370-58488310ea21-kube-api-access-8nw6s\") pod \"f6f971ee-83ba-4a5e-b370-58488310ea21\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") "
Sep 15 13:05:21.380938 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.380939    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-dshm\") pod \"f6f971ee-83ba-4a5e-b370-58488310ea21\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") "
Sep 15 13:05:21.381219 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.380989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f971ee-83ba-4a5e-b370-58488310ea21-tls-certs\") pod \"f6f971ee-83ba-4a5e-b370-58488310ea21\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") "
Sep 15 13:05:21.381219 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381070    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-home\") pod \"f6f971ee-83ba-4a5e-b370-58488310ea21\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") "
Sep 15 13:05:21.381219 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381097    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-tmp-dir\") pod \"f6f971ee-83ba-4a5e-b370-58488310ea21\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") "
Sep 15 13:05:21.381373 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381236    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-model-cache\") pod \"f6f971ee-83ba-4a5e-b370-58488310ea21\" (UID: \"f6f971ee-83ba-4a5e-b370-58488310ea21\") "
Sep 15 13:05:21.381425 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381350    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-home" (OuterVolumeSpecName: "home") pod "f6f971ee-83ba-4a5e-b370-58488310ea21" (UID: "f6f971ee-83ba-4a5e-b370-58488310ea21"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:21.381480 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381424    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f6f971ee-83ba-4a5e-b370-58488310ea21" (UID: "f6f971ee-83ba-4a5e-b370-58488310ea21"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:21.381535 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381487    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-model-cache" (OuterVolumeSpecName: "model-cache") pod "f6f971ee-83ba-4a5e-b370-58488310ea21" (UID: "f6f971ee-83ba-4a5e-b370-58488310ea21"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:21.381668 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381648    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-model-cache\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:21.381763 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381670    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-home\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:21.381763 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.381678    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-tmp-dir\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:21.383326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.383297    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6f971ee-83ba-4a5e-b370-58488310ea21-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f6f971ee-83ba-4a5e-b370-58488310ea21" (UID: "f6f971ee-83ba-4a5e-b370-58488310ea21"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:05:21.383566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.383545    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-dshm" (OuterVolumeSpecName: "dshm") pod "f6f971ee-83ba-4a5e-b370-58488310ea21" (UID: "f6f971ee-83ba-4a5e-b370-58488310ea21"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:21.383627 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.383572    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6f971ee-83ba-4a5e-b370-58488310ea21-kube-api-access-8nw6s" (OuterVolumeSpecName: "kube-api-access-8nw6s") pod "f6f971ee-83ba-4a5e-b370-58488310ea21" (UID: "f6f971ee-83ba-4a5e-b370-58488310ea21"). InnerVolumeSpecName "kube-api-access-8nw6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:05:21.482471 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.482434    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f971ee-83ba-4a5e-b370-58488310ea21-tls-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:21.482471 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.482463    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nw6s\" (UniqueName: \"kubernetes.io/projected/f6f971ee-83ba-4a5e-b370-58488310ea21-kube-api-access-8nw6s\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:21.482471 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.482475    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f971ee-83ba-4a5e-b370-58488310ea21-dshm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:21.583536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.583456    2562 generic.go:356] "Generic (PLEG): container finished" podID="f6f971ee-83ba-4a5e-b370-58488310ea21" containerID="4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538" exitCode=0
Sep 15 13:05:21.583536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.583499    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"
Sep 15 13:05:21.583711 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.583545    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm" event={"ID":"f6f971ee-83ba-4a5e-b370-58488310ea21","Type":"ContainerDied","Data":"4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538"}
Sep 15 13:05:21.583711 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.583591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm" event={"ID":"f6f971ee-83ba-4a5e-b370-58488310ea21","Type":"ContainerDied","Data":"ea81846005329cfdc3eb33f4115e7fbb092c7c53350390fd7ecc78b4d2738103"}
Sep 15 13:05:21.583711 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.583613    2562 scope.go:117] "RemoveContainer" containerID="4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538"
Sep 15 13:05:21.593641 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.593625    2562 scope.go:117] "RemoveContainer" containerID="4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538"
Sep 15 13:05:21.593927 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:05:21.593909    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538\": container with ID starting with 4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538 not found: ID does not exist" containerID="4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538"
Sep 15 13:05:21.593989 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.593939    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538"} err="failed to get container status \"4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538\": rpc error: code = NotFound desc = could not find container \"4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538\": container with ID starting with 4aa03caeb20b99c61fc28b4fba8f02d8d39bb41ff1ccc1b6c5d21a465eaf7538 not found: ID does not exist"
Sep 15 13:05:21.607754 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.607725    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"]
Sep 15 13:05:21.614328 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:21.614305    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-528f6b/test-model-528f6b-kserve-6d7f7774c5-whpnm"]
Sep 15 13:05:22.756191 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:22.756163    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6f971ee-83ba-4a5e-b370-58488310ea21" path="/var/lib/kubelet/pods/f6f971ee-83ba-4a5e-b370-58488310ea21/volumes"
Sep 15 13:05:37.649474 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.649402    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"]
Sep 15 13:05:37.760229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.760194    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"]
Sep 15 13:05:37.760662 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.760649    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6f971ee-83ba-4a5e-b370-58488310ea21" containerName="main"
Sep 15 13:05:37.760723 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.760664    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6f971ee-83ba-4a5e-b370-58488310ea21" containerName="main"
Sep 15 13:05:37.760723 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.760673    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3287deb5-7448-43eb-9724-6a3b0142b6df" containerName="main"
Sep 15 13:05:37.760723 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.760679    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3287deb5-7448-43eb-9724-6a3b0142b6df" containerName="main"
Sep 15 13:05:37.760820 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.760763    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6f971ee-83ba-4a5e-b370-58488310ea21" containerName="main"
Sep 15 13:05:37.760820 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.760771    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3287deb5-7448-43eb-9724-6a3b0142b6df" containerName="main"
Sep 15 13:05:37.765161 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.765134    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:37.767842 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.767816    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:05:37.767988 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.767857    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:05:37.767988 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.767924    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:05:37.772759 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.772737    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"]
Sep 15 13:05:37.852001 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.851975    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a70eb796-d33a-443e-83b1-5b305f145179-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s\" (UID: \"a70eb796-d33a-443e-83b1-5b305f145179\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:37.852170 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.852061    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbmvr\" (UniqueName: \"kubernetes.io/projected/a70eb796-d33a-443e-83b1-5b305f145179-kube-api-access-qbmvr\") pod \"maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s\" (UID: \"a70eb796-d33a-443e-83b1-5b305f145179\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:37.952738 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.952700    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbmvr\" (UniqueName: \"kubernetes.io/projected/a70eb796-d33a-443e-83b1-5b305f145179-kube-api-access-qbmvr\") pod \"maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s\" (UID: \"a70eb796-d33a-443e-83b1-5b305f145179\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:37.952950 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.952931    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a70eb796-d33a-443e-83b1-5b305f145179-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s\" (UID: \"a70eb796-d33a-443e-83b1-5b305f145179\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:37.953551 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.953530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a70eb796-d33a-443e-83b1-5b305f145179-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s\" (UID: \"a70eb796-d33a-443e-83b1-5b305f145179\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:37.960869 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:37.960851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbmvr\" (UniqueName: \"kubernetes.io/projected/a70eb796-d33a-443e-83b1-5b305f145179-kube-api-access-qbmvr\") pod \"maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s\" (UID: \"a70eb796-d33a-443e-83b1-5b305f145179\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:38.078259 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:38.078228    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:38.354141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:38.354113    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"]
Sep 15 13:05:38.356348 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:05:38.356310    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70eb796_d33a_443e_83b1_5b305f145179.slice/crio-d9db03131880656d640d935e2c4d94a390fb0b57f728c58ea70a5e08936b0afa WatchSource:0}: Error finding container d9db03131880656d640d935e2c4d94a390fb0b57f728c58ea70a5e08936b0afa: Status 404 returned error can't find the container with id d9db03131880656d640d935e2c4d94a390fb0b57f728c58ea70a5e08936b0afa
Sep 15 13:05:38.698932 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:38.698895    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s" event={"ID":"a70eb796-d33a-443e-83b1-5b305f145179","Type":"ContainerStarted","Data":"d2aa677382c9c804005c62f078f27fd11ae8d82eea9623b0a495d79a6e6197dc"}
Sep 15 13:05:38.699300 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:38.698982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s" event={"ID":"a70eb796-d33a-443e-83b1-5b305f145179","Type":"ContainerStarted","Data":"d9db03131880656d640d935e2c4d94a390fb0b57f728c58ea70a5e08936b0afa"}
Sep 15 13:05:38.717363 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:38.717308    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s" podStartSLOduration=1.7172891 podStartE2EDuration="1.7172891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:05:38.71683995 +0000 UTC m=+1256.509541681" watchObservedRunningTime="2026-09-15 13:05:38.7172891 +0000 UTC m=+1256.509990833"
Sep 15 13:05:39.706731 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:39.706691    2562 generic.go:356] "Generic (PLEG): container finished" podID="a70eb796-d33a-443e-83b1-5b305f145179" containerID="d2aa677382c9c804005c62f078f27fd11ae8d82eea9623b0a495d79a6e6197dc" exitCode=0
Sep 15 13:05:39.707120 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:39.706773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s" event={"ID":"a70eb796-d33a-443e-83b1-5b305f145179","Type":"ContainerDied","Data":"d2aa677382c9c804005c62f078f27fd11ae8d82eea9623b0a495d79a6e6197dc"}
Sep 15 13:05:40.850021 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:40.849997    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:40.988540 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:40.988447    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbmvr\" (UniqueName: \"kubernetes.io/projected/a70eb796-d33a-443e-83b1-5b305f145179-kube-api-access-qbmvr\") pod \"a70eb796-d33a-443e-83b1-5b305f145179\" (UID: \"a70eb796-d33a-443e-83b1-5b305f145179\") "
Sep 15 13:05:40.988712 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:40.988538    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a70eb796-d33a-443e-83b1-5b305f145179-maas-api-service-ca\") pod \"a70eb796-d33a-443e-83b1-5b305f145179\" (UID: \"a70eb796-d33a-443e-83b1-5b305f145179\") "
Sep 15 13:05:40.988877 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:40.988853    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a70eb796-d33a-443e-83b1-5b305f145179-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a70eb796-d33a-443e-83b1-5b305f145179" (UID: "a70eb796-d33a-443e-83b1-5b305f145179"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:05:40.989001 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:40.988984    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a70eb796-d33a-443e-83b1-5b305f145179-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:40.990579 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:40.990546    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a70eb796-d33a-443e-83b1-5b305f145179-kube-api-access-qbmvr" (OuterVolumeSpecName: "kube-api-access-qbmvr") pod "a70eb796-d33a-443e-83b1-5b305f145179" (UID: "a70eb796-d33a-443e-83b1-5b305f145179"). InnerVolumeSpecName "kube-api-access-qbmvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:05:41.090311 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:41.090282    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbmvr\" (UniqueName: \"kubernetes.io/projected/a70eb796-d33a-443e-83b1-5b305f145179-kube-api-access-qbmvr\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:41.721185 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:41.721149    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"
Sep 15 13:05:41.721387 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:41.721161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s" event={"ID":"a70eb796-d33a-443e-83b1-5b305f145179","Type":"ContainerDied","Data":"d9db03131880656d640d935e2c4d94a390fb0b57f728c58ea70a5e08936b0afa"}
Sep 15 13:05:41.721387 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:41.721270    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9db03131880656d640d935e2c4d94a390fb0b57f728c58ea70a5e08936b0afa"
Sep 15 13:05:42.822340 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:42.822304    2562 scope.go:117] "RemoveContainer" containerID="562df64bfd1af5efc1491ed132800fa2bc7a5b1d8b1efbb1f7373b08f7c01ab0"
Sep 15 13:05:42.831903 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:42.831881    2562 scope.go:117] "RemoveContainer" containerID="c30a1857ee964685473e8586c2a373ed063300a041e00cc8eb67fce26d35bf7a"
Sep 15 13:05:43.289616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:43.289580    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:05:52.668804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:52.668740    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk" podUID="0152c575-0a44-474c-9d4a-eda58f4aedb1" containerName="main" containerID="cri-o://995b478b8d040a6441bb2abba9f7ab71354e5b192564075a3a70b54a33fbe658" gracePeriod=45
Sep 15 13:05:52.786620 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:52.786593    2562 generic.go:356] "Generic (PLEG): container finished" podID="0152c575-0a44-474c-9d4a-eda58f4aedb1" containerID="995b478b8d040a6441bb2abba9f7ab71354e5b192564075a3a70b54a33fbe658" exitCode=0
Sep 15 13:05:52.786740 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:52.786661    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk" event={"ID":"0152c575-0a44-474c-9d4a-eda58f4aedb1","Type":"ContainerDied","Data":"995b478b8d040a6441bb2abba9f7ab71354e5b192564075a3a70b54a33fbe658"}
Sep 15 13:05:52.913234 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:52.913206    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:05:53.016584 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016556    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-home\") pod \"0152c575-0a44-474c-9d4a-eda58f4aedb1\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") "
Sep 15 13:05:53.016744 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016609    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-tmp-dir\") pod \"0152c575-0a44-474c-9d4a-eda58f4aedb1\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") "
Sep 15 13:05:53.016744 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016627    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-model-cache\") pod \"0152c575-0a44-474c-9d4a-eda58f4aedb1\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") "
Sep 15 13:05:53.016744 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016708    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0152c575-0a44-474c-9d4a-eda58f4aedb1-tls-certs\") pod \"0152c575-0a44-474c-9d4a-eda58f4aedb1\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") "
Sep 15 13:05:53.016861 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016761    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-dshm\") pod \"0152c575-0a44-474c-9d4a-eda58f4aedb1\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") "
Sep 15 13:05:53.016861 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016819    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xslq9\" (UniqueName: \"kubernetes.io/projected/0152c575-0a44-474c-9d4a-eda58f4aedb1-kube-api-access-xslq9\") pod \"0152c575-0a44-474c-9d4a-eda58f4aedb1\" (UID: \"0152c575-0a44-474c-9d4a-eda58f4aedb1\") "
Sep 15 13:05:53.016973 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016857    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-home" (OuterVolumeSpecName: "home") pod "0152c575-0a44-474c-9d4a-eda58f4aedb1" (UID: "0152c575-0a44-474c-9d4a-eda58f4aedb1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:53.016973 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016864    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0152c575-0a44-474c-9d4a-eda58f4aedb1" (UID: "0152c575-0a44-474c-9d4a-eda58f4aedb1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:53.016973 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.016950    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-model-cache" (OuterVolumeSpecName: "model-cache") pod "0152c575-0a44-474c-9d4a-eda58f4aedb1" (UID: "0152c575-0a44-474c-9d4a-eda58f4aedb1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:53.017170 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.017147    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-home\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:53.017170 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.017159    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-tmp-dir\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:53.017170 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.017168    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-model-cache\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:53.019003 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.018981    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-dshm" (OuterVolumeSpecName: "dshm") pod "0152c575-0a44-474c-9d4a-eda58f4aedb1" (UID: "0152c575-0a44-474c-9d4a-eda58f4aedb1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:05:53.019003 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.018995    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0152c575-0a44-474c-9d4a-eda58f4aedb1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0152c575-0a44-474c-9d4a-eda58f4aedb1" (UID: "0152c575-0a44-474c-9d4a-eda58f4aedb1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:05:53.019138 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.019118    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0152c575-0a44-474c-9d4a-eda58f4aedb1-kube-api-access-xslq9" (OuterVolumeSpecName: "kube-api-access-xslq9") pod "0152c575-0a44-474c-9d4a-eda58f4aedb1" (UID: "0152c575-0a44-474c-9d4a-eda58f4aedb1"). InnerVolumeSpecName "kube-api-access-xslq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:05:53.118615 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.118588    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0152c575-0a44-474c-9d4a-eda58f4aedb1-dshm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:53.118615 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.118611    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xslq9\" (UniqueName: \"kubernetes.io/projected/0152c575-0a44-474c-9d4a-eda58f4aedb1-kube-api-access-xslq9\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:53.118799 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.118622    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0152c575-0a44-474c-9d4a-eda58f4aedb1-tls-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:05:53.794862 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.794837    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"
Sep 15 13:05:53.795300 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.794833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk" event={"ID":"0152c575-0a44-474c-9d4a-eda58f4aedb1","Type":"ContainerDied","Data":"303dd30a5f3b7155315b67a2743bbff10d421747cd11c020aa3c1d356ff4612b"}
Sep 15 13:05:53.795300 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.794965    2562 scope.go:117] "RemoveContainer" containerID="995b478b8d040a6441bb2abba9f7ab71354e5b192564075a3a70b54a33fbe658"
Sep 15 13:05:53.817102 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.817079    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"]
Sep 15 13:05:53.821284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:53.821263    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1c35dd/test-model-1c35dd-kserve-8fc8d4849-dsbsk"]
Sep 15 13:05:54.756292 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:05:54.756257    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0152c575-0a44-474c-9d4a-eda58f4aedb1" path="/var/lib/kubelet/pods/0152c575-0a44-474c-9d4a-eda58f4aedb1/volumes"
Sep 15 13:06:42.866899 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:06:42.866871    2562 scope.go:117] "RemoveContainer" containerID="6bc308594d30ecb149190f665f83d7c97f25433c6c994a5850a72b24d4942347"
Sep 15 13:06:42.875200 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:06:42.875132    2562 scope.go:117] "RemoveContainer" containerID="615aa60ced7dad7b47a32af98d52427629ea77e3e9fcd932681f68b760f968bc"
Sep 15 13:06:42.884217 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:06:42.884196    2562 scope.go:117] "RemoveContainer" containerID="f885a454e6d83224f5fd8c5319af2aa231b264f9079912e2cafcf2d99acd933b"
Sep 15 13:07:42.922839 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:07:42.922807    2562 scope.go:117] "RemoveContainer" containerID="d8e4ab55efc822fefe6200ed3480122c895bc06fd82212c242eaa4dc4e8f9b08"
Sep 15 13:07:42.934161 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:07:42.934132    2562 scope.go:117] "RemoveContainer" containerID="51b5236c8d8db4de9f0b4a2aab8d8d329cc306436df54e9b062217cb2b8f8935"
Sep 15 13:08:16.253079 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.253042    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"]
Sep 15 13:08:16.253539 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.253498    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0152c575-0a44-474c-9d4a-eda58f4aedb1" containerName="main"
Sep 15 13:08:16.253539 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.253512    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0152c575-0a44-474c-9d4a-eda58f4aedb1" containerName="main"
Sep 15 13:08:16.253539 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.253534    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a70eb796-d33a-443e-83b1-5b305f145179" containerName="revoke-keys"
Sep 15 13:08:16.253539 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.253540    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a70eb796-d33a-443e-83b1-5b305f145179" containerName="revoke-keys"
Sep 15 13:08:16.253675 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.253618    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0152c575-0a44-474c-9d4a-eda58f4aedb1" containerName="main"
Sep 15 13:08:16.253675 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.253626    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a70eb796-d33a-443e-83b1-5b305f145179" containerName="revoke-keys"
Sep 15 13:08:16.256857 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.256837    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.260310 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.260292    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-de540b29-openshift-default-dockercfg-nhm6h\""
Sep 15 13:08:16.288100 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.288072    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"]
Sep 15 13:08:16.314770 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.314728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-data\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.314978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.314789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-credential-socket\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.314978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.314823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hpxq\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-kube-api-access-6hpxq\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.314978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.314870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-certs\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.315194 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.315005    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-socket\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.315194 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.315098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c874a553-cbab-4c45-aa05-95aa42799325-istiod-ca-cert\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.315194 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.315138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-istio-token\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.315353 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.315198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-envoy\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.315353 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.315273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c874a553-cbab-4c45-aa05-95aa42799325-istio-podinfo\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.415849 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.415810    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c874a553-cbab-4c45-aa05-95aa42799325-istio-podinfo\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416056 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.415880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-data\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416056 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.415902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-credential-socket\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416056 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.415923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hpxq\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-kube-api-access-6hpxq\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416056 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.415955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-certs\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416314 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.416205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-socket\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416314 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.416261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c874a553-cbab-4c45-aa05-95aa42799325-istiod-ca-cert\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416314 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.416299    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-istio-token\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416468 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.416369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-envoy\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416468 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.416449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-data\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.416486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-credential-socket\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.416528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-certs\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.416661 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.416611    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-socket\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.417059 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.417018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c874a553-cbab-4c45-aa05-95aa42799325-istiod-ca-cert\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.418527 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.418509    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-envoy\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.418751 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.418733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c874a553-cbab-4c45-aa05-95aa42799325-istio-podinfo\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.425246 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.425218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-istio-token\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.425546 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.425524    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hpxq\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-kube-api-access-6hpxq\") pod \"e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") " pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.567000 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.566908    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:16.884878 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.884851    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"]
Sep 15 13:08:16.886949 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:08:16.886916    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc874a553_cbab_4c45_aa05_95aa42799325.slice/crio-7dc85e92f4ad0aa2cc24b679518e38d227793e46992b20194770b4955fc672e6 WatchSource:0}: Error finding container 7dc85e92f4ad0aa2cc24b679518e38d227793e46992b20194770b4955fc672e6: Status 404 returned error can't find the container with id 7dc85e92f4ad0aa2cc24b679518e38d227793e46992b20194770b4955fc672e6
Sep 15 13:08:16.889515 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.889473    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:08:16.889619 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.889599    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:08:16.889672 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:16.889650    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:08:17.624677 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:17.624636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" event={"ID":"c874a553-cbab-4c45-aa05-95aa42799325","Type":"ContainerStarted","Data":"f97b0499536a51a0392512a9d7612c449618de83e2537842170bd4de2fdfb0a7"}
Sep 15 13:08:17.625086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:17.624704    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" event={"ID":"c874a553-cbab-4c45-aa05-95aa42799325","Type":"ContainerStarted","Data":"7dc85e92f4ad0aa2cc24b679518e38d227793e46992b20194770b4955fc672e6"}
Sep 15 13:08:17.650048 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:17.649982    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" podStartSLOduration=1.649965873 podStartE2EDuration="1.649965873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:08:17.648971663 +0000 UTC m=+1415.441673396" watchObservedRunningTime="2026-09-15 13:08:17.649965873 +0000 UTC m=+1415.442667604"
Sep 15 13:08:18.567643 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:18.567611    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:18.572343 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:18.572318    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:18.631947 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:18.631922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:18.632963 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:18.632944    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:36.783529 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:36.783447    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:08:47.804809 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:47.804772    2562 generic.go:356] "Generic (PLEG): container finished" podID="c874a553-cbab-4c45-aa05-95aa42799325" containerID="f97b0499536a51a0392512a9d7612c449618de83e2537842170bd4de2fdfb0a7" exitCode=0
Sep 15 13:08:47.805301 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:47.804840    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" event={"ID":"c874a553-cbab-4c45-aa05-95aa42799325","Type":"ContainerDied","Data":"f97b0499536a51a0392512a9d7612c449618de83e2537842170bd4de2fdfb0a7"}
Sep 15 13:08:47.805370 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:47.805309    2562 scope.go:117] "RemoveContainer" containerID="f97b0499536a51a0392512a9d7612c449618de83e2537842170bd4de2fdfb0a7"
Sep 15 13:08:47.806503 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:47.806473    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:08:47.806613 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:47.806591    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:08:47.806680 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:47.806645    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:08:48.632660 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:48.632606    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:48.813054 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:48.812996    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" event={"ID":"c874a553-cbab-4c45-aa05-95aa42799325","Type":"ContainerStarted","Data":"a428e6bb4237cbcb74f577eb2836e491a84756b5d231c4f47d6b4a22091ec853"}
Sep 15 13:08:49.567236 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:49.567197    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:49.568559 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:49.568528    2562 patch_prober.go:28] interesting pod/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 15 13:08:49.568688 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:49.568579    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 15 13:08:50.567518 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:50.567481    2562 patch_prober.go:28] interesting pod/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 15 13:08:50.567918 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:50.567533    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 15 13:08:51.567434 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:51.567399    2562 patch_prober.go:28] interesting pod/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 15 13:08:51.567690 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:51.567666    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 15 13:08:52.568151 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:52.568113    2562 patch_prober.go:28] interesting pod/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 15 13:08:52.568530 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:52.568180    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 15 13:08:53.567819 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:53.567770    2562 patch_prober.go:28] interesting pod/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 15 13:08:53.568022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:53.567859    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 15 13:08:54.458537 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.458501    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"]
Sep 15 13:08:54.461512 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.461493    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:54.464216 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.464167    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:08:54.464409 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.464298    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:08:54.465425 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.465409    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:08:54.473137 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.473116    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"]
Sep 15 13:08:54.525907 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.525878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6tdl\" (UniqueName: \"kubernetes.io/projected/17d5659a-6717-4f6e-a23a-04cf70e517d5-kube-api-access-d6tdl\") pod \"maas-api-revoke-keys-e2e-mt-de540b29-bgqdn\" (UID: \"17d5659a-6717-4f6e-a23a-04cf70e517d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:54.526049 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.525925    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d5659a-6717-4f6e-a23a-04cf70e517d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-de540b29-bgqdn\" (UID: \"17d5659a-6717-4f6e-a23a-04cf70e517d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:54.571773 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.571750    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:54.572209 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.572192    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:54.572685 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.572669    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:08:54.627368 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.627333    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6tdl\" (UniqueName: \"kubernetes.io/projected/17d5659a-6717-4f6e-a23a-04cf70e517d5-kube-api-access-d6tdl\") pod \"maas-api-revoke-keys-e2e-mt-de540b29-bgqdn\" (UID: \"17d5659a-6717-4f6e-a23a-04cf70e517d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:54.627523 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.627408    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d5659a-6717-4f6e-a23a-04cf70e517d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-de540b29-bgqdn\" (UID: \"17d5659a-6717-4f6e-a23a-04cf70e517d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:54.628415 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.628181    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d5659a-6717-4f6e-a23a-04cf70e517d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-de540b29-bgqdn\" (UID: \"17d5659a-6717-4f6e-a23a-04cf70e517d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:54.638903 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.638875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6tdl\" (UniqueName: \"kubernetes.io/projected/17d5659a-6717-4f6e-a23a-04cf70e517d5-kube-api-access-d6tdl\") pod \"maas-api-revoke-keys-e2e-mt-de540b29-bgqdn\" (UID: \"17d5659a-6717-4f6e-a23a-04cf70e517d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:54.774211 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:54.774125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:55.054267 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:55.054244    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"]
Sep 15 13:08:55.056135 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:08:55.056108    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17d5659a_6717_4f6e_a23a_04cf70e517d5.slice/crio-b1e96dfbc5c688afecaa9cbddfae1a6f960b0c6452eef1e49652c4db4d164a5a WatchSource:0}: Error finding container b1e96dfbc5c688afecaa9cbddfae1a6f960b0c6452eef1e49652c4db4d164a5a: Status 404 returned error can't find the container with id b1e96dfbc5c688afecaa9cbddfae1a6f960b0c6452eef1e49652c4db4d164a5a
Sep 15 13:08:55.865380 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:55.865342    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn" event={"ID":"17d5659a-6717-4f6e-a23a-04cf70e517d5","Type":"ContainerStarted","Data":"dd58b3f95d398f2a7710cdfee1c58917e316e70bae94e7e979dbb4ff3ab83e38"}
Sep 15 13:08:55.865757 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:55.865404    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn" event={"ID":"17d5659a-6717-4f6e-a23a-04cf70e517d5","Type":"ContainerStarted","Data":"b1e96dfbc5c688afecaa9cbddfae1a6f960b0c6452eef1e49652c4db4d164a5a"}
Sep 15 13:08:55.882281 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:55.882223    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn" podStartSLOduration=1.882205011 podStartE2EDuration="1.882205011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:08:55.881305163 +0000 UTC m=+1453.674006894" watchObservedRunningTime="2026-09-15 13:08:55.882205011 +0000 UTC m=+1453.674906742"
Sep 15 13:08:56.873012 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:56.872976    2562 generic.go:356] "Generic (PLEG): container finished" podID="17d5659a-6717-4f6e-a23a-04cf70e517d5" containerID="dd58b3f95d398f2a7710cdfee1c58917e316e70bae94e7e979dbb4ff3ab83e38" exitCode=0
Sep 15 13:08:56.873420 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:56.873065    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn" event={"ID":"17d5659a-6717-4f6e-a23a-04cf70e517d5","Type":"ContainerDied","Data":"dd58b3f95d398f2a7710cdfee1c58917e316e70bae94e7e979dbb4ff3ab83e38"}
Sep 15 13:08:58.024752 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.024723    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:58.166688 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.166583    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6tdl\" (UniqueName: \"kubernetes.io/projected/17d5659a-6717-4f6e-a23a-04cf70e517d5-kube-api-access-d6tdl\") pod \"17d5659a-6717-4f6e-a23a-04cf70e517d5\" (UID: \"17d5659a-6717-4f6e-a23a-04cf70e517d5\") "
Sep 15 13:08:58.166688 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.166640    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d5659a-6717-4f6e-a23a-04cf70e517d5-maas-api-service-ca\") pod \"17d5659a-6717-4f6e-a23a-04cf70e517d5\" (UID: \"17d5659a-6717-4f6e-a23a-04cf70e517d5\") "
Sep 15 13:08:58.167073 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.167012    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17d5659a-6717-4f6e-a23a-04cf70e517d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17d5659a-6717-4f6e-a23a-04cf70e517d5" (UID: "17d5659a-6717-4f6e-a23a-04cf70e517d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:08:58.167178 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.167130    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d5659a-6717-4f6e-a23a-04cf70e517d5-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:08:58.168813 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.168785    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17d5659a-6717-4f6e-a23a-04cf70e517d5-kube-api-access-d6tdl" (OuterVolumeSpecName: "kube-api-access-d6tdl") pod "17d5659a-6717-4f6e-a23a-04cf70e517d5" (UID: "17d5659a-6717-4f6e-a23a-04cf70e517d5"). InnerVolumeSpecName "kube-api-access-d6tdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:08:58.267937 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.267899    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6tdl\" (UniqueName: \"kubernetes.io/projected/17d5659a-6717-4f6e-a23a-04cf70e517d5-kube-api-access-d6tdl\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:08:58.889682 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.889657    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"
Sep 15 13:08:58.889860 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.889694    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn" event={"ID":"17d5659a-6717-4f6e-a23a-04cf70e517d5","Type":"ContainerDied","Data":"b1e96dfbc5c688afecaa9cbddfae1a6f960b0c6452eef1e49652c4db4d164a5a"}
Sep 15 13:08:58.889860 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:08:58.889748    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1e96dfbc5c688afecaa9cbddfae1a6f960b0c6452eef1e49652c4db4d164a5a"
Sep 15 13:09:05.168537 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:05.168502    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"]
Sep 15 13:09:05.170683 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:05.170657    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de540b29-bgqdn"]
Sep 15 13:09:06.762377 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:06.762342    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17d5659a-6717-4f6e-a23a-04cf70e517d5" path="/var/lib/kubelet/pods/17d5659a-6717-4f6e-a23a-04cf70e517d5/volumes"
Sep 15 13:09:11.763022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:11.762990    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"]
Sep 15 13:09:11.763491 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:11.763240    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy" containerID="cri-o://a428e6bb4237cbcb74f577eb2836e491a84756b5d231c4f47d6b4a22091ec853" gracePeriod=30
Sep 15 13:09:11.889073 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:11.889017    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:09:16.996995 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:16.996964    2562 generic.go:356] "Generic (PLEG): container finished" podID="c874a553-cbab-4c45-aa05-95aa42799325" containerID="a428e6bb4237cbcb74f577eb2836e491a84756b5d231c4f47d6b4a22091ec853" exitCode=0
Sep 15 13:09:16.997395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:16.997049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" event={"ID":"c874a553-cbab-4c45-aa05-95aa42799325","Type":"ContainerDied","Data":"a428e6bb4237cbcb74f577eb2836e491a84756b5d231c4f47d6b4a22091ec853"}
Sep 15 13:09:16.997395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:16.997119    2562 scope.go:117] "RemoveContainer" containerID="f97b0499536a51a0392512a9d7612c449618de83e2537842170bd4de2fdfb0a7"
Sep 15 13:09:17.027149 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.027128    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:09:17.073396 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073364    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-envoy\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.073573 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073416    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-credential-socket\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.073573 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073470    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-data\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.073573 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073495    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-socket\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.073573 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073552    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c874a553-cbab-4c45-aa05-95aa42799325-istiod-ca-cert\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.073804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073583    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-certs\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.073804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073623    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c874a553-cbab-4c45-aa05-95aa42799325-istio-podinfo\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.073804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073652    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hpxq\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-kube-api-access-6hpxq\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.073804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073708    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-istio-token\") pod \"c874a553-cbab-4c45-aa05-95aa42799325\" (UID: \"c874a553-cbab-4c45-aa05-95aa42799325\") "
Sep 15 13:09:17.074015 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073647    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:09:17.074015 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073801    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-data" (OuterVolumeSpecName: "istio-data") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:09:17.074015 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073761    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:09:17.074015 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.073857    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:09:17.074284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.074184    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:17.074284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.074184    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c874a553-cbab-4c45-aa05-95aa42799325-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:09:17.074284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.074204    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:17.074284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.074220    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:17.074284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.074235    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:17.076127 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.076103    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c874a553-cbab-4c45-aa05-95aa42799325-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:09:17.076127 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.076114    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:09:17.076362 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.076337    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-kube-api-access-6hpxq" (OuterVolumeSpecName: "kube-api-access-6hpxq") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "kube-api-access-6hpxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:09:17.076455 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.076434    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-istio-token" (OuterVolumeSpecName: "istio-token") pod "c874a553-cbab-4c45-aa05-95aa42799325" (UID: "c874a553-cbab-4c45-aa05-95aa42799325"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:09:17.175373 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.175338    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:17.175373 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.175367    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c874a553-cbab-4c45-aa05-95aa42799325-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:17.175554 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.175381    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c874a553-cbab-4c45-aa05-95aa42799325-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:17.175554 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.175393    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c874a553-cbab-4c45-aa05-95aa42799325-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:17.175554 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:17.175404    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hpxq\" (UniqueName: \"kubernetes.io/projected/c874a553-cbab-4c45-aa05-95aa42799325-kube-api-access-6hpxq\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:09:18.004200 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:18.004172    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"
Sep 15 13:09:18.004669 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:18.004173    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f" event={"ID":"c874a553-cbab-4c45-aa05-95aa42799325","Type":"ContainerDied","Data":"7dc85e92f4ad0aa2cc24b679518e38d227793e46992b20194770b4955fc672e6"}
Sep 15 13:09:18.004669 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:18.004320    2562 scope.go:117] "RemoveContainer" containerID="a428e6bb4237cbcb74f577eb2836e491a84756b5d231c4f47d6b4a22091ec853"
Sep 15 13:09:18.039230 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:18.039197    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"]
Sep 15 13:09:18.044910 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:18.044887    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-de540b29-openshift-default-7bf4777f68-q5v8f"]
Sep 15 13:09:18.756769 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:18.756736    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c874a553-cbab-4c45-aa05-95aa42799325" path="/var/lib/kubelet/pods/c874a553-cbab-4c45-aa05-95aa42799325/volumes"
Sep 15 13:09:25.944187 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944150    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"]
Sep 15 13:09:25.944791 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944770    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy"
Sep 15 13:09:25.944873 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944794    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy"
Sep 15 13:09:25.944873 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944817    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy"
Sep 15 13:09:25.944873 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944826    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy"
Sep 15 13:09:25.944873 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944845    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17d5659a-6717-4f6e-a23a-04cf70e517d5" containerName="revoke-keys"
Sep 15 13:09:25.944873 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944854    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="17d5659a-6717-4f6e-a23a-04cf70e517d5" containerName="revoke-keys"
Sep 15 13:09:25.945155 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944985    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy"
Sep 15 13:09:25.945155 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.944997    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c874a553-cbab-4c45-aa05-95aa42799325" containerName="istio-proxy"
Sep 15 13:09:25.945155 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.945011    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="17d5659a-6717-4f6e-a23a-04cf70e517d5" containerName="revoke-keys"
Sep 15 13:09:25.948566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.948545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:25.952059 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.952025    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-f6a966-openshift-default-dockercfg-8lt4w\""
Sep 15 13:09:25.960938 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:25.960910    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"]
Sep 15 13:09:26.065513 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.065466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-envoy\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.065813 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.065784    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48pxw\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-kube-api-access-48pxw\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.066023 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.066002    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-data\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.066193 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.066176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-token\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.066303 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.066288    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-podinfo\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.066479 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.066456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3462c55f-8088-478c-b6f1-e270b3059fd4-istiod-ca-cert\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.066585 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.066573    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-credential-socket\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.066725 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.066709    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-socket\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.066870 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.066838    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-certs\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.168511 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-token\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.168511 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-podinfo\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.168785 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3462c55f-8088-478c-b6f1-e270b3059fd4-istiod-ca-cert\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.168785 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168571    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-credential-socket\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.168785 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-socket\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.168785 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168628    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-certs\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.168785 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-envoy\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.168785 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48pxw\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-kube-api-access-48pxw\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.169116 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.168789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-data\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.169116 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.169104    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-socket\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.169226 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.169159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-credential-socket\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.169226 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.169190    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-certs\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.169443 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.169419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3462c55f-8088-478c-b6f1-e270b3059fd4-istiod-ca-cert\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.169508 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.169443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-data\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.171005 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.170987    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-envoy\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.171266 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.171247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-podinfo\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.178171 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.178144    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-token\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.178387 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.178263    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48pxw\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-kube-api-access-48pxw\") pod \"e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") " pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.259968 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.259923    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:26.539946 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.539921    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"]
Sep 15 13:09:26.541801 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:09:26.541763    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3462c55f_8088_478c_b6f1_e270b3059fd4.slice/crio-c6c55b0b4295e2152b5f88207dfd5ca078b00fd9aace638e3b1164e92a0f0b4a WatchSource:0}: Error finding container c6c55b0b4295e2152b5f88207dfd5ca078b00fd9aace638e3b1164e92a0f0b4a: Status 404 returned error can't find the container with id c6c55b0b4295e2152b5f88207dfd5ca078b00fd9aace638e3b1164e92a0f0b4a
Sep 15 13:09:26.544161 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.544132    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:09:26.544243 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.544191    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:09:26.544243 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:26.544223    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:09:27.059266 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:27.059228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" event={"ID":"3462c55f-8088-478c-b6f1-e270b3059fd4","Type":"ContainerStarted","Data":"ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847"}
Sep 15 13:09:27.059656 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:27.059292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" event={"ID":"3462c55f-8088-478c-b6f1-e270b3059fd4","Type":"ContainerStarted","Data":"c6c55b0b4295e2152b5f88207dfd5ca078b00fd9aace638e3b1164e92a0f0b4a"}
Sep 15 13:09:27.085955 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:27.085901    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podStartSLOduration=2.085885903 podStartE2EDuration="2.085885903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:09:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:09:27.083025175 +0000 UTC m=+1484.875726908" watchObservedRunningTime="2026-09-15 13:09:27.085885903 +0000 UTC m=+1484.878587634"
Sep 15 13:09:27.260548 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:27.260506    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:27.265760 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:27.265722    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:28.065224 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:28.065195    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:28.066217 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:28.066197    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:42.809076 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:42.809017    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:09:42.812484 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:42.812457    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:09:42.812975 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:42.812949    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:09:42.816344 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:42.816323    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:09:42.966155 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:42.966115    2562 scope.go:117] "RemoveContainer" containerID="9650d94926b3ab49700b36e4cc39b7c5b4dca08ffd792d1a5c1e42b02cb14af3"
Sep 15 13:09:43.685284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:43.685251    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:09:54.226572 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:54.226481    2562 generic.go:356] "Generic (PLEG): container finished" podID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerID="ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847" exitCode=0
Sep 15 13:09:54.226572 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:54.226509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" event={"ID":"3462c55f-8088-478c-b6f1-e270b3059fd4","Type":"ContainerDied","Data":"ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847"}
Sep 15 13:09:54.227054 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:54.227012    2562 scope.go:117] "RemoveContainer" containerID="ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847"
Sep 15 13:09:54.228486 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:54.228453    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:09:54.228572 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:54.228540    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:09:54.228610 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:54.228584    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:09:55.244642 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:55.244604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" event={"ID":"3462c55f-8088-478c-b6f1-e270b3059fd4","Type":"ContainerStarted","Data":"30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9"}
Sep 15 13:09:55.261124 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:55.261089    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:55.262353 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:55.262330    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 13:09:55.262435 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:55.262390    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 13:09:56.260797 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:56.260765    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 13:09:56.261222 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:56.260820    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 13:09:57.260920 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:57.260890    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 13:09:57.261351 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:57.260952    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 13:09:58.065061 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:58.065018    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:09:58.261330 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:58.261293    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 13:09:58.261725 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:58.261345    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 13:09:59.261245 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.261207    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 13:09:59.261496 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.261271    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 13:09:59.569056 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.568953    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"]
Sep 15 13:09:59.572535 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.572516    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.575709 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.575679    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-ddd0eb-openshift-default-dockercfg-9gmgs\""
Sep 15 13:09:59.583467 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.583438    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"]
Sep 15 13:09:59.742536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.742489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-data\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.742873 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.742847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjsjd\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-kube-api-access-cjsjd\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.743022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.743006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-envoy\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.743187 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.743170    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istiod-ca-cert\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.743358 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.743341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-token\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.743497 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.743481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-socket\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.743628 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.743613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-certs\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.743791 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.743774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-credential-socket\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.743922 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.743905    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-podinfo\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.845758 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.845643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-data\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.845758 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.845740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjsjd\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-kube-api-access-cjsjd\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.845995 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.845779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-envoy\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.845995 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.845819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istiod-ca-cert\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.845995 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.845886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-token\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.845995 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.845923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-socket\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.845995 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.845956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-certs\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.846274 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.846050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-credential-socket\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.846274 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.846076    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-podinfo\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.846935 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.846903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-socket\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.847068 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.847018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istiod-ca-cert\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.847332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.847303    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-data\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.847750 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.847725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-certs\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.847910 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.847868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-credential-socket\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.849381 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.849294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-podinfo\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.851310 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.851288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-envoy\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.856025 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.855998    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-token\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.856590 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.856568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjsjd\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-kube-api-access-cjsjd\") pod \"e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") " pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:09:59.886186 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:09:59.886164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:00.192123 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:00.192070    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"]
Sep 15 13:10:00.194705 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:10:00.194641    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4278035_22b6_4d22_8ee3_5ee6cdc19ab8.slice/crio-881664af0af41a7857035e3c3adcf3fc616c2786d7a6b265877fc229a0ec8104 WatchSource:0}: Error finding container 881664af0af41a7857035e3c3adcf3fc616c2786d7a6b265877fc229a0ec8104: Status 404 returned error can't find the container with id 881664af0af41a7857035e3c3adcf3fc616c2786d7a6b265877fc229a0ec8104
Sep 15 13:10:00.197191 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:00.197154    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:00.197284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:00.197243    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:00.197330 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:00.197293    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:00.260672 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:00.260640    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 13:10:00.260868 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:00.260701    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 13:10:00.279164 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:00.279137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" event={"ID":"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8","Type":"ContainerStarted","Data":"881664af0af41a7857035e3c3adcf3fc616c2786d7a6b265877fc229a0ec8104"}
Sep 15 13:10:01.265004 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:01.264969    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:10:01.265890 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:01.265871    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:10:01.286749 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:01.286660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" event={"ID":"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8","Type":"ContainerStarted","Data":"4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16"}
Sep 15 13:10:01.319897 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:01.319847    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podStartSLOduration=2.319829771 podStartE2EDuration="2.319829771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:10:01.318011379 +0000 UTC m=+1519.110713112" watchObservedRunningTime="2026-09-15 13:10:01.319829771 +0000 UTC m=+1519.112531502"
Sep 15 13:10:01.886705 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:01.886670    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:01.891952 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:01.891928    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:02.293106 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:02.293076    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:02.294090 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:02.294067    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:08.342826 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.342738    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"]
Sep 15 13:10:08.343843 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.343778    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy" containerID="cri-o://8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68" gracePeriod=30
Sep 15 13:10:08.673662 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.673631    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"]
Sep 15 13:10:08.676372 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.676352    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:08.679200 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.679174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:10:08.679313 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.679262    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:10:08.679637 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.679617    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:10:08.686946 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.686923    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"]
Sep 15 13:10:08.741491 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.741458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl\" (UID: \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:08.741663 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.741506    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfjkb\" (UniqueName: \"kubernetes.io/projected/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-kube-api-access-vfjkb\") pod \"maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl\" (UID: \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:08.843138 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.843099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl\" (UID: \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:08.843372 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.843347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfjkb\" (UniqueName: \"kubernetes.io/projected/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-kube-api-access-vfjkb\") pod \"maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl\" (UID: \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:08.843778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.843754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl\" (UID: \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:08.852564 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.852540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfjkb\" (UniqueName: \"kubernetes.io/projected/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-kube-api-access-vfjkb\") pod \"maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl\" (UID: \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:08.989210 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:08.989104    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:09.264420 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:09.264390    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"]
Sep 15 13:10:09.269419 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:10:09.269367    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5619f5_f511_48fb_98ed_f1b00b5f6f83.slice/crio-24cc62c4ae1e15ee43508dc2dd26f8a912fb4a9025810b1442af0b01f392651d WatchSource:0}: Error finding container 24cc62c4ae1e15ee43508dc2dd26f8a912fb4a9025810b1442af0b01f392651d: Status 404 returned error can't find the container with id 24cc62c4ae1e15ee43508dc2dd26f8a912fb4a9025810b1442af0b01f392651d
Sep 15 13:10:09.341879 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:09.341843    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl" event={"ID":"ab5619f5-f511-48fb-98ed-f1b00b5f6f83","Type":"ContainerStarted","Data":"24cc62c4ae1e15ee43508dc2dd26f8a912fb4a9025810b1442af0b01f392651d"}
Sep 15 13:10:10.350441 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:10.350406    2562 generic.go:356] "Generic (PLEG): container finished" podID="ab5619f5-f511-48fb-98ed-f1b00b5f6f83" containerID="ef25b984137edbc0ff154159447d7e141c816f76fe57757be9370b60372f49de" exitCode=0
Sep 15 13:10:10.350823 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:10.350484    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl" event={"ID":"ab5619f5-f511-48fb-98ed-f1b00b5f6f83","Type":"ContainerDied","Data":"ef25b984137edbc0ff154159447d7e141c816f76fe57757be9370b60372f49de"}
Sep 15 13:10:11.501742 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:11.501716    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:11.573464 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:11.573428    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfjkb\" (UniqueName: \"kubernetes.io/projected/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-kube-api-access-vfjkb\") pod \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\" (UID: \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\") "
Sep 15 13:10:11.573464 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:11.573466    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-maas-api-service-ca\") pod \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\" (UID: \"ab5619f5-f511-48fb-98ed-f1b00b5f6f83\") "
Sep 15 13:10:11.573934 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:11.573907    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab5619f5-f511-48fb-98ed-f1b00b5f6f83" (UID: "ab5619f5-f511-48fb-98ed-f1b00b5f6f83"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:10:11.574071 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:11.573992    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:11.575697 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:11.575676    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-kube-api-access-vfjkb" (OuterVolumeSpecName: "kube-api-access-vfjkb") pod "ab5619f5-f511-48fb-98ed-f1b00b5f6f83" (UID: "ab5619f5-f511-48fb-98ed-f1b00b5f6f83"). InnerVolumeSpecName "kube-api-access-vfjkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:10:11.675258 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:11.675202    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfjkb\" (UniqueName: \"kubernetes.io/projected/ab5619f5-f511-48fb-98ed-f1b00b5f6f83-kube-api-access-vfjkb\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:12.365682 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:12.365636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl" event={"ID":"ab5619f5-f511-48fb-98ed-f1b00b5f6f83","Type":"ContainerDied","Data":"24cc62c4ae1e15ee43508dc2dd26f8a912fb4a9025810b1442af0b01f392651d"}
Sep 15 13:10:12.365899 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:12.365710    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24cc62c4ae1e15ee43508dc2dd26f8a912fb4a9025810b1442af0b01f392651d"
Sep 15 13:10:12.365899 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:12.365809    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"
Sep 15 13:10:13.607440 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.607367    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 13:10:13.696383 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696348    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-token\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.696578 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696406    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-envoy\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.696578 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696435    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-data\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.696578 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696463    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istiod-ca-cert\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.696578 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696546    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-credential-socket\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.696578 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696571    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-socket\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.696890 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696606    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-podinfo\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.696890 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696724    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-certs\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.696890 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696760    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlbq5\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-kube-api-access-nlbq5\") pod \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\" (UID: \"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c\") "
Sep 15 13:10:13.697097 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696753    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:10:13.697169 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696800    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:10:13.697169 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696963    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:10:13.697169 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.696988    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:10:13.697169 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.697094    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:10:13.697395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.697260    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:13.697395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.697279    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:13.697395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.697294    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:13.697395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.697309    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:13.697395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.697321    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:13.698898 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.698872    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:10:13.699024 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.698946    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-kube-api-access-nlbq5" (OuterVolumeSpecName: "kube-api-access-nlbq5") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "kube-api-access-nlbq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:10:13.699120 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.699101    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:10:13.699174 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.699160    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" (UID: "41ec43ad-25ce-49f1-9a67-d6c1880c3f9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:10:13.798250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.798215    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:13.798250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.798245    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlbq5\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-kube-api-access-nlbq5\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:13.798250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.798257    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:13.798498 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:13.798268    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:14.379004 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.378970    2562 generic.go:356] "Generic (PLEG): container finished" podID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerID="8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68" exitCode=0
Sep 15 13:10:14.379217 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.379072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" event={"ID":"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c","Type":"ContainerDied","Data":"8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68"}
Sep 15 13:10:14.379217 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.379150    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq" event={"ID":"41ec43ad-25ce-49f1-9a67-d6c1880c3f9c","Type":"ContainerDied","Data":"f09747bea3bb4b594530c71759bd8bae6242f5edaf497563f59f0ced5181ca77"}
Sep 15 13:10:14.379217 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.379080    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"
Sep 15 13:10:14.379386 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.379172    2562 scope.go:117] "RemoveContainer" containerID="8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68"
Sep 15 13:10:14.389579 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.389560    2562 scope.go:117] "RemoveContainer" containerID="354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27"
Sep 15 13:10:14.398090 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.398011    2562 scope.go:117] "RemoveContainer" containerID="8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68"
Sep 15 13:10:14.398378 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:14.398357    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68\": container with ID starting with 8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68 not found: ID does not exist" containerID="8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68"
Sep 15 13:10:14.398437 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.398387    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68"} err="failed to get container status \"8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68\": rpc error: code = NotFound desc = could not find container \"8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68\": container with ID starting with 8992c332a0f1c7a7b8425f0455553ab75523e300e3a531f0991368a198fb7f68 not found: ID does not exist"
Sep 15 13:10:14.398437 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.398405    2562 scope.go:117] "RemoveContainer" containerID="354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27"
Sep 15 13:10:14.398665 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:14.398646    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27\": container with ID starting with 354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27 not found: ID does not exist" containerID="354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27"
Sep 15 13:10:14.398706 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.398671    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27"} err="failed to get container status \"354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27\": rpc error: code = NotFound desc = could not find container \"354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27\": container with ID starting with 354b3cfa83c3b233869b78760b4bbc2cd7bf0747d7994f7567bbcb6714a6ae27 not found: ID does not exist"
Sep 15 13:10:14.404699 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.404676    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"]
Sep 15 13:10:14.410461 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.410435    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-528f6b-openshift-default-56c5d6f6c6-rvmmq"]
Sep 15 13:10:14.757173 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:14.757141    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" path="/var/lib/kubelet/pods/41ec43ad-25ce-49f1-9a67-d6c1880c3f9c/volumes"
Sep 15 13:10:18.634149 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.634105    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7768-57b91a"]
Sep 15 13:10:18.634577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.634564    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5619f5-f511-48fb-98ed-f1b00b5f6f83" containerName="revoke-keys"
Sep 15 13:10:18.634622 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.634578    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5619f5-f511-48fb-98ed-f1b00b5f6f83" containerName="revoke-keys"
Sep 15 13:10:18.634622 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.634597    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy"
Sep 15 13:10:18.634622 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.634603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy"
Sep 15 13:10:18.634622 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.634613    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy"
Sep 15 13:10:18.634622 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.634618    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy"
Sep 15 13:10:18.635146 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.635119    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy"
Sep 15 13:10:18.635146 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.635148    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab5619f5-f511-48fb-98ed-f1b00b5f6f83" containerName="revoke-keys"
Sep 15 13:10:18.635315 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.635218    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ec43ad-25ce-49f1-9a67-d6c1880c3f9c" containerName="istio-proxy"
Sep 15 13:10:18.640635 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.640608    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:18.644107 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.644084    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8hhg6\""
Sep 15 13:10:18.646995 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.646967    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7768-57b91a"]
Sep 15 13:10:18.752224 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.752192    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj4f8\" (UniqueName: \"kubernetes.io/projected/e0b1dae9-4d0d-4524-970d-88155650aa9f-kube-api-access-cj4f8\") pod \"test-curl-7768-57b91a\" (UID: \"e0b1dae9-4d0d-4524-970d-88155650aa9f\") " pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:18.853552 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.853508    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4f8\" (UniqueName: \"kubernetes.io/projected/e0b1dae9-4d0d-4524-970d-88155650aa9f-kube-api-access-cj4f8\") pod \"test-curl-7768-57b91a\" (UID: \"e0b1dae9-4d0d-4524-970d-88155650aa9f\") " pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:18.862876 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.862844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj4f8\" (UniqueName: \"kubernetes.io/projected/e0b1dae9-4d0d-4524-970d-88155650aa9f-kube-api-access-cj4f8\") pod \"test-curl-7768-57b91a\" (UID: \"e0b1dae9-4d0d-4524-970d-88155650aa9f\") " pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:18.954605 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:18.954571    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:19.238890 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:19.238864    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7768-57b91a"]
Sep 15 13:10:19.240848 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:10:19.240821    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0b1dae9_4d0d_4524_970d_88155650aa9f.slice/crio-c3de7b0ee19797604de0fd1faea789db3b40106d69e5e0471e242d101e679208 WatchSource:0}: Error finding container c3de7b0ee19797604de0fd1faea789db3b40106d69e5e0471e242d101e679208: Status 404 returned error can't find the container with id c3de7b0ee19797604de0fd1faea789db3b40106d69e5e0471e242d101e679208
Sep 15 13:10:19.242973 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:19.242952    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 13:10:19.410327 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:19.410292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7768-57b91a" event={"ID":"e0b1dae9-4d0d-4524-970d-88155650aa9f","Type":"ContainerStarted","Data":"c3de7b0ee19797604de0fd1faea789db3b40106d69e5e0471e242d101e679208"}
Sep 15 13:10:20.417347 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:20.417306    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7768-57b91a" event={"ID":"e0b1dae9-4d0d-4524-970d-88155650aa9f","Type":"ContainerStarted","Data":"1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3"}
Sep 15 13:10:20.433041 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:20.432979    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7768-57b91a" podStartSLOduration=1.467803342 podStartE2EDuration="2.432964193s" totalImagesPullingTime="965.160851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:10:20.432210394 +0000 UTC m=+1538.224912125" watchObservedRunningTime="2026-09-15 13:10:20.432964193 +0000 UTC m=+1538.225665923"
Sep 15 13:10:21.046089 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.046051    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7768-57b91a"]
Sep 15 13:10:21.049521 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.049487    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7768-57b91a"]
Sep 15 13:10:21.162169 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.162136    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7768-b9bfb8"]
Sep 15 13:10:21.162589 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.162576    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0b1dae9-4d0d-4524-970d-88155650aa9f" containerName="test-curl-7768-57b91a"
Sep 15 13:10:21.162634 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.162591    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0b1dae9-4d0d-4524-970d-88155650aa9f" containerName="test-curl-7768-57b91a"
Sep 15 13:10:21.162681 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.162672    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0b1dae9-4d0d-4524-970d-88155650aa9f" containerName="test-curl-7768-57b91a"
Sep 15 13:10:21.164919 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.164903    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:21.173064 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.173022    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7768-b9bfb8"]
Sep 15 13:10:21.277813 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.277776    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbvs\" (UniqueName: \"kubernetes.io/projected/f2461c1d-7ea7-4956-9584-fe53644c5de1-kube-api-access-msbvs\") pod \"test-curl-7768-b9bfb8\" (UID: \"f2461c1d-7ea7-4956-9584-fe53644c5de1\") " pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:21.379131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.379013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbvs\" (UniqueName: \"kubernetes.io/projected/f2461c1d-7ea7-4956-9584-fe53644c5de1-kube-api-access-msbvs\") pod \"test-curl-7768-b9bfb8\" (UID: \"f2461c1d-7ea7-4956-9584-fe53644c5de1\") " pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:21.388252 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.388219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbvs\" (UniqueName: \"kubernetes.io/projected/f2461c1d-7ea7-4956-9584-fe53644c5de1-kube-api-access-msbvs\") pod \"test-curl-7768-b9bfb8\" (UID: \"f2461c1d-7ea7-4956-9584-fe53644c5de1\") " pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:21.423121 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:21.423085    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7768-57b91a" container="test-curl-7768-57b91a"
Sep 15 13:10:21.475111 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.475079    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:21.749366 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:21.749341    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7768-b9bfb8"]
Sep 15 13:10:21.751327 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:10:21.751298    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2461c1d_7ea7_4956_9584_fe53644c5de1.slice/crio-834e372e2a9b93af648cb1f76da3c9af5fc3f96844ddc85838472aa110c3b6c6 WatchSource:0}: Error finding container 834e372e2a9b93af648cb1f76da3c9af5fc3f96844ddc85838472aa110c3b6c6: Status 404 returned error can't find the container with id 834e372e2a9b93af648cb1f76da3c9af5fc3f96844ddc85838472aa110c3b6c6
Sep 15 13:10:22.431640 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:22.431602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7768-b9bfb8" event={"ID":"f2461c1d-7ea7-4956-9584-fe53644c5de1","Type":"ContainerStarted","Data":"3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b"}
Sep 15 13:10:22.432201 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:22.431652    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7768-b9bfb8" event={"ID":"f2461c1d-7ea7-4956-9584-fe53644c5de1","Type":"ContainerStarted","Data":"834e372e2a9b93af648cb1f76da3c9af5fc3f96844ddc85838472aa110c3b6c6"}
Sep 15 13:10:22.432201 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:22.431685    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7768-57b91a" podUID="e0b1dae9-4d0d-4524-970d-88155650aa9f" containerName="test-curl-7768-57b91a" containerID="cri-o://1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3" gracePeriod=2
Sep 15 13:10:22.447449 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:22.447396    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7768-b9bfb8" podStartSLOduration=1.447381632 podStartE2EDuration="1.447381632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:10:22.44622852 +0000 UTC m=+1540.238930251" watchObservedRunningTime="2026-09-15 13:10:22.447381632 +0000 UTC m=+1540.240083363"
Sep 15 13:10:22.448199 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:22.448172    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-57b91a\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="e0b1dae9-4d0d-4524-970d-88155650aa9f" pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:22.449931 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:22.449905    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-57b91a\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="e0b1dae9-4d0d-4524-970d-88155650aa9f" pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:22.756266 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:22.756154    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-57b91a\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="e0b1dae9-4d0d-4524-970d-88155650aa9f" pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:22.888666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:22.888632    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:10:23.039680 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.039599    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7768-b9bfb8"]
Sep 15 13:10:23.043955 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.043926    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7768-b9bfb8"]
Sep 15 13:10:23.153433 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.153405    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7768-d4e5c4"]
Sep 15 13:10:23.153992 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.153966    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2461c1d-7ea7-4956-9584-fe53644c5de1" containerName="test-curl-7768-b9bfb8"
Sep 15 13:10:23.153992 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.153992    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2461c1d-7ea7-4956-9584-fe53644c5de1" containerName="test-curl-7768-b9bfb8"
Sep 15 13:10:23.154206 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.154090    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2461c1d-7ea7-4956-9584-fe53644c5de1" containerName="test-curl-7768-b9bfb8"
Sep 15 13:10:23.156449 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.156427    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:23.163716 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.163692    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7768-d4e5c4"]
Sep 15 13:10:23.299520 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.299437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltskt\" (UniqueName: \"kubernetes.io/projected/2eb7c6cc-c064-4be0-aa20-8cd8c38794ec-kube-api-access-ltskt\") pod \"test-curl-7768-d4e5c4\" (UID: \"2eb7c6cc-c064-4be0-aa20-8cd8c38794ec\") " pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:23.400744 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.400711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltskt\" (UniqueName: \"kubernetes.io/projected/2eb7c6cc-c064-4be0-aa20-8cd8c38794ec-kube-api-access-ltskt\") pod \"test-curl-7768-d4e5c4\" (UID: \"2eb7c6cc-c064-4be0-aa20-8cd8c38794ec\") " pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:23.410026 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.409990    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltskt\" (UniqueName: \"kubernetes.io/projected/2eb7c6cc-c064-4be0-aa20-8cd8c38794ec-kube-api-access-ltskt\") pod \"test-curl-7768-d4e5c4\" (UID: \"2eb7c6cc-c064-4be0-aa20-8cd8c38794ec\") " pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:23.438256 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:23.438229    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7768-b9bfb8" container="test-curl-7768-b9bfb8"
Sep 15 13:10:23.467122 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.467095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:23.737770 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:23.737727    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7768-d4e5c4"]
Sep 15 13:10:23.739317 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:10:23.739296    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2eb7c6cc_c064_4be0_aa20_8cd8c38794ec.slice/crio-bc67d8159d0421e294026fdc3daa2e29bf31e4d5e204478cc93dda606b902cc8 WatchSource:0}: Error finding container bc67d8159d0421e294026fdc3daa2e29bf31e4d5e204478cc93dda606b902cc8: Status 404 returned error can't find the container with id bc67d8159d0421e294026fdc3daa2e29bf31e4d5e204478cc93dda606b902cc8
Sep 15 13:10:24.446881 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.446783    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7768-d4e5c4" event={"ID":"2eb7c6cc-c064-4be0-aa20-8cd8c38794ec","Type":"ContainerStarted","Data":"ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4"}
Sep 15 13:10:24.446881 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.446831    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7768-d4e5c4" event={"ID":"2eb7c6cc-c064-4be0-aa20-8cd8c38794ec","Type":"ContainerStarted","Data":"bc67d8159d0421e294026fdc3daa2e29bf31e4d5e204478cc93dda606b902cc8"}
Sep 15 13:10:24.446881 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.446828    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7768-b9bfb8" podUID="f2461c1d-7ea7-4956-9584-fe53644c5de1" containerName="test-curl-7768-b9bfb8" containerID="cri-o://3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b" gracePeriod=2
Sep 15 13:10:24.449103 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:24.449076    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-b9bfb8\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="f2461c1d-7ea7-4956-9584-fe53644c5de1" pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:24.451148 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:24.451115    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-b9bfb8\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="f2461c1d-7ea7-4956-9584-fe53644c5de1" pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:24.471927 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.471856    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7768-d4e5c4" podStartSLOduration=1.471835531 podStartE2EDuration="1.471835531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:10:24.467928311 +0000 UTC m=+1542.260630043" watchObservedRunningTime="2026-09-15 13:10:24.471835531 +0000 UTC m=+1542.264537263"
Sep 15 13:10:24.629807 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.629783    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:24.632041 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:24.632011    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-b9bfb8\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="f2461c1d-7ea7-4956-9584-fe53644c5de1" pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:24.713131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.713052    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj4f8\" (UniqueName: \"kubernetes.io/projected/e0b1dae9-4d0d-4524-970d-88155650aa9f-kube-api-access-cj4f8\") pod \"e0b1dae9-4d0d-4524-970d-88155650aa9f\" (UID: \"e0b1dae9-4d0d-4524-970d-88155650aa9f\") "
Sep 15 13:10:24.714959 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.714925    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0b1dae9-4d0d-4524-970d-88155650aa9f-kube-api-access-cj4f8" (OuterVolumeSpecName: "kube-api-access-cj4f8") pod "e0b1dae9-4d0d-4524-970d-88155650aa9f" (UID: "e0b1dae9-4d0d-4524-970d-88155650aa9f"). InnerVolumeSpecName "kube-api-access-cj4f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:10:24.756739 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.756709    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0b1dae9-4d0d-4524-970d-88155650aa9f" path="/var/lib/kubelet/pods/e0b1dae9-4d0d-4524-970d-88155650aa9f/volumes"
Sep 15 13:10:24.814260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:24.814241    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj4f8\" (UniqueName: \"kubernetes.io/projected/e0b1dae9-4d0d-4524-970d-88155650aa9f-kube-api-access-cj4f8\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:25.027046 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.026947    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7768-d4e5c4"]
Sep 15 13:10:25.031949 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.031919    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7768-d4e5c4"]
Sep 15 13:10:25.143208 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.143170    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7768-59b4b3"]
Sep 15 13:10:25.143692 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.143673    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" containerName="test-curl-7768-d4e5c4"
Sep 15 13:10:25.143787 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.143694    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" containerName="test-curl-7768-d4e5c4"
Sep 15 13:10:25.143849 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.143797    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" containerName="test-curl-7768-d4e5c4"
Sep 15 13:10:25.146197 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.146177    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:25.155633 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.155608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7768-59b4b3"]
Sep 15 13:10:25.218570 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.218535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzwgm\" (UniqueName: \"kubernetes.io/projected/cc3c3f5c-e669-4af3-abf9-cde65cf8394c-kube-api-access-lzwgm\") pod \"test-curl-7768-59b4b3\" (UID: \"cc3c3f5c-e669-4af3-abf9-cde65cf8394c\") " pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:25.319961 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.319873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzwgm\" (UniqueName: \"kubernetes.io/projected/cc3c3f5c-e669-4af3-abf9-cde65cf8394c-kube-api-access-lzwgm\") pod \"test-curl-7768-59b4b3\" (UID: \"cc3c3f5c-e669-4af3-abf9-cde65cf8394c\") " pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:25.329449 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.329414    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzwgm\" (UniqueName: \"kubernetes.io/projected/cc3c3f5c-e669-4af3-abf9-cde65cf8394c-kube-api-access-lzwgm\") pod \"test-curl-7768-59b4b3\" (UID: \"cc3c3f5c-e669-4af3-abf9-cde65cf8394c\") " pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:25.455146 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.455102    2562 generic.go:356] "Generic (PLEG): container finished" podID="e0b1dae9-4d0d-4524-970d-88155650aa9f" containerID="1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3" exitCode=137
Sep 15 13:10:25.455146 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.455150    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-57b91a"
Sep 15 13:10:25.455558 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.455205    2562 scope.go:117] "RemoveContainer" containerID="1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3"
Sep 15 13:10:25.455558 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:25.455475    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7768-d4e5c4" container="test-curl-7768-d4e5c4"
Sep 15 13:10:25.455681 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.455628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:25.466635 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.466610    2562 scope.go:117] "RemoveContainer" containerID="1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3"
Sep 15 13:10:25.466959 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:25.466929    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3\": container with ID starting with 1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3 not found: ID does not exist" containerID="1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3"
Sep 15 13:10:25.467058 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.466971    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3"} err="failed to get container status \"1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3\": rpc error: code = NotFound desc = could not find container \"1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3\": container with ID starting with 1f70b0b755b456bedba7fa83e1838f614200da202b0801902a5a8499ed55cfa3 not found: ID does not exist"
Sep 15 13:10:25.733990 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:25.733947    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7768-59b4b3"]
Sep 15 13:10:25.745597 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:10:25.745560    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc3c3f5c_e669_4af3_abf9_cde65cf8394c.slice/crio-a18c8c57fda8d742dba9be10d7b64e621ca339c04325575dcc5dda8b59deb72e WatchSource:0}: Error finding container a18c8c57fda8d742dba9be10d7b64e621ca339c04325575dcc5dda8b59deb72e: Status 404 returned error can't find the container with id a18c8c57fda8d742dba9be10d7b64e621ca339c04325575dcc5dda8b59deb72e
Sep 15 13:10:26.472057 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:26.472006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7768-59b4b3" event={"ID":"cc3c3f5c-e669-4af3-abf9-cde65cf8394c","Type":"ContainerStarted","Data":"4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7"}
Sep 15 13:10:26.472413 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:26.472070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7768-59b4b3" event={"ID":"cc3c3f5c-e669-4af3-abf9-cde65cf8394c","Type":"ContainerStarted","Data":"a18c8c57fda8d742dba9be10d7b64e621ca339c04325575dcc5dda8b59deb72e"}
Sep 15 13:10:26.472413 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:26.472075    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7768-d4e5c4" podUID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" containerName="test-curl-7768-d4e5c4" containerID="cri-o://ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4" gracePeriod=2
Sep 15 13:10:26.474771 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:26.474746    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-d4e5c4\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:26.491269 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:26.491243    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-d4e5c4\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:26.491381 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:26.491322    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7768-59b4b3" podStartSLOduration=1.491307868 podStartE2EDuration="1.491307868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:10:26.489292504 +0000 UTC m=+1544.281994234" watchObservedRunningTime="2026-09-15 13:10:26.491307868 +0000 UTC m=+1544.284009598"
Sep 15 13:10:26.681472 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:26.681448    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:26.683845 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:26.683822    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-d4e5c4\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:26.853444 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:26.853365    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msbvs\" (UniqueName: \"kubernetes.io/projected/f2461c1d-7ea7-4956-9584-fe53644c5de1-kube-api-access-msbvs\") pod \"f2461c1d-7ea7-4956-9584-fe53644c5de1\" (UID: \"f2461c1d-7ea7-4956-9584-fe53644c5de1\") "
Sep 15 13:10:26.855423 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:26.855399    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2461c1d-7ea7-4956-9584-fe53644c5de1-kube-api-access-msbvs" (OuterVolumeSpecName: "kube-api-access-msbvs") pod "f2461c1d-7ea7-4956-9584-fe53644c5de1" (UID: "f2461c1d-7ea7-4956-9584-fe53644c5de1"). InnerVolumeSpecName "kube-api-access-msbvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:10:26.954597 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:26.954554    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msbvs\" (UniqueName: \"kubernetes.io/projected/f2461c1d-7ea7-4956-9584-fe53644c5de1-kube-api-access-msbvs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:27.155065 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:27.154965    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7768-59b4b3"]
Sep 15 13:10:27.158942 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:27.158916    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7768-59b4b3"]
Sep 15 13:10:27.479264 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:27.479232    2562 generic.go:356] "Generic (PLEG): container finished" podID="f2461c1d-7ea7-4956-9584-fe53644c5de1" containerID="3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b" exitCode=137
Sep 15 13:10:27.479671 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:27.479273    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-b9bfb8"
Sep 15 13:10:27.479671 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:27.479331    2562 scope.go:117] "RemoveContainer" containerID="3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b"
Sep 15 13:10:27.479671 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:27.479539    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7768-59b4b3" container="test-curl-7768-59b4b3"
Sep 15 13:10:27.488786 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:27.488766    2562 scope.go:117] "RemoveContainer" containerID="3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b"
Sep 15 13:10:27.489132 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:27.489108    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b\": container with ID starting with 3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b not found: ID does not exist" containerID="3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b"
Sep 15 13:10:27.489191 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:27.489147    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b"} err="failed to get container status \"3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b\": rpc error: code = NotFound desc = could not find container \"3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b\": container with ID starting with 3be52604a268717f5c7072ce0179cfe37326e46119505e6b63c0b65d3eece85b not found: ID does not exist"
Sep 15 13:10:28.000069 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.000020    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"]
Sep 15 13:10:28.000579 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.000561    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc3c3f5c-e669-4af3-abf9-cde65cf8394c" containerName="test-curl-7768-59b4b3"
Sep 15 13:10:28.000666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.000582    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc3c3f5c-e669-4af3-abf9-cde65cf8394c" containerName="test-curl-7768-59b4b3"
Sep 15 13:10:28.000717 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.000678    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc3c3f5c-e669-4af3-abf9-cde65cf8394c" containerName="test-curl-7768-59b4b3"
Sep 15 13:10:28.003735 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.003705    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.006588 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.006569    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7e508e61-openshift-default-dockercfg-qkn5r\""
Sep 15 13:10:28.016438 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.016412    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"]
Sep 15 13:10:28.167942 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.167907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-token\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.168131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.167951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-certs\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.168131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.168050    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58bbef12-2135-43f6-90e3-71cf9f6a538f-istiod-ca-cert\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.168131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.168107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-credential-socket\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.168287 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.168175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-envoy\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.168287 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.168242    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4jll\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-kube-api-access-d4jll\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.168287 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.168284    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-podinfo\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.168379 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.168309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-data\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.168379 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.168359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-socket\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269136 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269024    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-token\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269136 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-certs\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269321 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58bbef12-2135-43f6-90e3-71cf9f6a538f-istiod-ca-cert\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269321 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-credential-socket\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269321 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269211    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-envoy\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269321 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4jll\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-kube-api-access-d4jll\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269508 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-podinfo\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269508 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269385    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-data\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269508 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-socket\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269657 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269529    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-certs\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269657 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269633    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-credential-socket\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269802 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-data\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269867 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-socket\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.269867 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.269858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58bbef12-2135-43f6-90e3-71cf9f6a538f-istiod-ca-cert\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.271721 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.271700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-envoy\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.271823 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.271793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-podinfo\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.277220 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.277192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-token\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.277323 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.277306    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4jll\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-kube-api-access-d4jll\") pod \"e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") " pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.315927 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.315900    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:28.487139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.487108    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7768-59b4b3" podUID="cc3c3f5c-e669-4af3-abf9-cde65cf8394c" containerName="test-curl-7768-59b4b3" containerID="cri-o://4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7" gracePeriod=2
Sep 15 13:10:28.489706 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:28.489675    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-59b4b3\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="cc3c3f5c-e669-4af3-abf9-cde65cf8394c" pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:28.617095 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.616998    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"]
Sep 15 13:10:28.626091 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:10:28.625136    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58bbef12_2135_43f6_90e3_71cf9f6a538f.slice/crio-2abd4d7ce630fae407cc5b4233a6c7896f51858cd60308df386e4d68e1f773f5 WatchSource:0}: Error finding container 2abd4d7ce630fae407cc5b4233a6c7896f51858cd60308df386e4d68e1f773f5: Status 404 returned error can't find the container with id 2abd4d7ce630fae407cc5b4233a6c7896f51858cd60308df386e4d68e1f773f5
Sep 15 13:10:28.651052 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.647370    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:28.651052 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.647461    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:28.651052 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.647506    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:28.693607 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.693584    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:28.695736 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:28.695704    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7768-59b4b3\" is forbidden: User \"system:node:ip-10-0-132-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-169.ec2.internal' and this object" podUID="cc3c3f5c-e669-4af3-abf9-cde65cf8394c" pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:28.758468 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.758434    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2461c1d-7ea7-4956-9584-fe53644c5de1" path="/var/lib/kubelet/pods/f2461c1d-7ea7-4956-9584-fe53644c5de1/volumes"
Sep 15 13:10:28.876667 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.876576    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltskt\" (UniqueName: \"kubernetes.io/projected/2eb7c6cc-c064-4be0-aa20-8cd8c38794ec-kube-api-access-ltskt\") pod \"2eb7c6cc-c064-4be0-aa20-8cd8c38794ec\" (UID: \"2eb7c6cc-c064-4be0-aa20-8cd8c38794ec\") "
Sep 15 13:10:28.878627 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.878594    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eb7c6cc-c064-4be0-aa20-8cd8c38794ec-kube-api-access-ltskt" (OuterVolumeSpecName: "kube-api-access-ltskt") pod "2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" (UID: "2eb7c6cc-c064-4be0-aa20-8cd8c38794ec"). InnerVolumeSpecName "kube-api-access-ltskt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:10:28.977906 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:28.977871    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltskt\" (UniqueName: \"kubernetes.io/projected/2eb7c6cc-c064-4be0-aa20-8cd8c38794ec-kube-api-access-ltskt\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:29.493853 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:29.493823    2562 generic.go:356] "Generic (PLEG): container finished" podID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" containerID="ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4" exitCode=137
Sep 15 13:10:29.494329 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:29.493862    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-d4e5c4"
Sep 15 13:10:29.494329 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:29.493928    2562 scope.go:117] "RemoveContainer" containerID="ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4"
Sep 15 13:10:29.496149 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:29.495978    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" event={"ID":"58bbef12-2135-43f6-90e3-71cf9f6a538f","Type":"ContainerStarted","Data":"e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe"}
Sep 15 13:10:29.496149 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:29.496063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" event={"ID":"58bbef12-2135-43f6-90e3-71cf9f6a538f","Type":"ContainerStarted","Data":"2abd4d7ce630fae407cc5b4233a6c7896f51858cd60308df386e4d68e1f773f5"}
Sep 15 13:10:29.508375 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:29.508353    2562 scope.go:117] "RemoveContainer" containerID="ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4"
Sep 15 13:10:29.508628 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:29.508608    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4\": container with ID starting with ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4 not found: ID does not exist" containerID="ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4"
Sep 15 13:10:29.508688 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:29.508634    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4"} err="failed to get container status \"ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4\": rpc error: code = NotFound desc = could not find container \"ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4\": container with ID starting with ad9b4615686dcec8cb3c62811f84ca7ff5abb3c365dc0de09e7fde08409f63f4 not found: ID does not exist"
Sep 15 13:10:29.518682 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:29.518636    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podStartSLOduration=2.51862225 podStartE2EDuration="2.51862225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:10:29.516568463 +0000 UTC m=+1547.309270196" watchObservedRunningTime="2026-09-15 13:10:29.51862225 +0000 UTC m=+1547.311323980"
Sep 15 13:10:30.316609 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.316572    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:30.326767 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.326728    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:30.503393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.503368    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:30.504267 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.504246    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:10:30.696353 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.696327    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:30.757853 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.757826    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eb7c6cc-c064-4be0-aa20-8cd8c38794ec" path="/var/lib/kubelet/pods/2eb7c6cc-c064-4be0-aa20-8cd8c38794ec/volumes"
Sep 15 13:10:30.798294 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.798264    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzwgm\" (UniqueName: \"kubernetes.io/projected/cc3c3f5c-e669-4af3-abf9-cde65cf8394c-kube-api-access-lzwgm\") pod \"cc3c3f5c-e669-4af3-abf9-cde65cf8394c\" (UID: \"cc3c3f5c-e669-4af3-abf9-cde65cf8394c\") "
Sep 15 13:10:30.800356 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.800331    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc3c3f5c-e669-4af3-abf9-cde65cf8394c-kube-api-access-lzwgm" (OuterVolumeSpecName: "kube-api-access-lzwgm") pod "cc3c3f5c-e669-4af3-abf9-cde65cf8394c" (UID: "cc3c3f5c-e669-4af3-abf9-cde65cf8394c"). InnerVolumeSpecName "kube-api-access-lzwgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:10:30.899531 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:30.899453    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzwgm\" (UniqueName: \"kubernetes.io/projected/cc3c3f5c-e669-4af3-abf9-cde65cf8394c-kube-api-access-lzwgm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:10:31.510119 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:31.510076    2562 generic.go:356] "Generic (PLEG): container finished" podID="cc3c3f5c-e669-4af3-abf9-cde65cf8394c" containerID="4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7" exitCode=137
Sep 15 13:10:31.510581 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:31.510122    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7768-59b4b3"
Sep 15 13:10:31.510581 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:31.510165    2562 scope.go:117] "RemoveContainer" containerID="4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7"
Sep 15 13:10:31.520331 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:31.520307    2562 scope.go:117] "RemoveContainer" containerID="4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7"
Sep 15 13:10:31.520649 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:31.520618    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7\": container with ID starting with 4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7 not found: ID does not exist" containerID="4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7"
Sep 15 13:10:31.520736 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:31.520663    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7"} err="failed to get container status \"4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7\": rpc error: code = NotFound desc = could not find container \"4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7\": container with ID starting with 4753ab84a636603068f664fabf3bb5676507c80e1489aa3cae67634bd96664d7 not found: ID does not exist"
Sep 15 13:10:32.758848 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:32.758807    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc3c3f5c-e669-4af3-abf9-cde65cf8394c" path="/var/lib/kubelet/pods/cc3c3f5c-e669-4af3-abf9-cde65cf8394c/volumes"
Sep 15 13:10:33.334192 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:33.334152    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 15 13:10:33.334380 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:33.334206    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 15 13:10:34.532777 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:34.532741    2562 generic.go:356] "Generic (PLEG): container finished" podID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerID="4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16" exitCode=0
Sep 15 13:10:34.533288 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:34.532815    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" event={"ID":"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8","Type":"ContainerDied","Data":"4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16"}
Sep 15 13:10:34.533354 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:34.533303    2562 scope.go:117] "RemoveContainer" containerID="4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16"
Sep 15 13:10:34.534486 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:34.534460    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:34.534548 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:34.534526    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:34.534586 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:34.534560    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:10:35.071748 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.071714    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"]
Sep 15 13:10:35.074426 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.074400    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.077292 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.077265    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7e508e61-serving-cert\""
Sep 15 13:10:35.077411 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.077331    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:10:35.077535 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.077516    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fdkj6\""
Sep 15 13:10:35.078779 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.078759    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:10:35.078893 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.078794    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 13:10:35.087717 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.087683    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"]
Sep 15 13:10:35.149651 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.149620    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e25595-9bc1-4925-866b-f545b2817819-openshift-service-ca\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.149651 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.149660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfgxs\" (UniqueName: \"kubernetes.io/projected/28e25595-9bc1-4925-866b-f545b2817819-kube-api-access-hfgxs\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.149916 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.149703    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.149916 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.149771    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-metrics-certs\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.250890 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.250854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e25595-9bc1-4925-866b-f545b2817819-openshift-service-ca\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.251090 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.250898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfgxs\" (UniqueName: \"kubernetes.io/projected/28e25595-9bc1-4925-866b-f545b2817819-kube-api-access-hfgxs\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.251090 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.251027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.251174 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.251113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-metrics-certs\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.251219 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:35.251169    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-serving-cert: secret "maas-api-e2e-mt-7e508e61-serving-cert" not found
Sep 15 13:10:35.251263 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:10:35.251239    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls podName:28e25595-9bc1-4925-866b-f545b2817819 nodeName:}" failed. No retries permitted until 2026-09-15 13:10:35.75121826 +0000 UTC m=+1553.543919989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls") pod "maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v" (UID: "28e25595-9bc1-4925-866b-f545b2817819") : secret "maas-api-e2e-mt-7e508e61-serving-cert" not found
Sep 15 13:10:35.256047 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.251956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e25595-9bc1-4925-866b-f545b2817819-openshift-service-ca\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.260508 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.260474    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-metrics-certs\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.262591 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.262550    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfgxs\" (UniqueName: \"kubernetes.io/projected/28e25595-9bc1-4925-866b-f545b2817819-kube-api-access-hfgxs\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.541680 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.541641    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" event={"ID":"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8","Type":"ContainerStarted","Data":"823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a"}
Sep 15 13:10:35.760056 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.759260    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.767106 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.767071    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls\") pod \"maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:35.887525 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.887429    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:35.887682 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.887571    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 15 13:10:35.887682 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.887622    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 15 13:10:35.988699 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:35.988669    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:36.290440 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:36.290409    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"]
Sep 15 13:10:36.292013 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:10:36.291989    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e25595_9bc1_4925_866b_f545b2817819.slice/crio-c2c2db70a704773201709bff1487191fc35c79437f96ff1c9fffc391cfff9b5a WatchSource:0}: Error finding container c2c2db70a704773201709bff1487191fc35c79437f96ff1c9fffc391cfff9b5a: Status 404 returned error can't find the container with id c2c2db70a704773201709bff1487191fc35c79437f96ff1c9fffc391cfff9b5a
Sep 15 13:10:36.552496 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:36.552414    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v" event={"ID":"28e25595-9bc1-4925-866b-f545b2817819","Type":"ContainerStarted","Data":"c2c2db70a704773201709bff1487191fc35c79437f96ff1c9fffc391cfff9b5a"}
Sep 15 13:10:36.887158 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:36.887067    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 15 13:10:36.887158 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:36.887137    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 15 13:10:37.886956 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:37.886924    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 15 13:10:37.887397 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:37.887001    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 15 13:10:38.886781 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:38.886744    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 15 13:10:38.886994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:38.886825    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 15 13:10:39.576253 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:39.576161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v" event={"ID":"28e25595-9bc1-4925-866b-f545b2817819","Type":"ContainerStarted","Data":"ffbc122e7ff8f7a16163fd5495a4d28e86c06e8478fb793b844b56748f33ebf2"}
Sep 15 13:10:39.576253 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:39.576220    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:39.599774 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:39.599714    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v" podStartSLOduration=1.6284935329999999 podStartE2EDuration="4.599698853s" totalImagesPullingTime="2.97120532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:10:39.597502834 +0000 UTC m=+1557.390204566" watchObservedRunningTime="2026-09-15 13:10:39.599698853 +0000 UTC m=+1557.392400583"
Sep 15 13:10:39.886887 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:39.886792    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 15 13:10:39.886887 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:39.886853    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 15 13:10:40.886805 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:40.886767    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 15 13:10:40.886994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:40.886826    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 15 13:10:41.031244 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:41.031208    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"]
Sep 15 13:10:41.035242 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:41.035186    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-528f6b-bg86s"]
Sep 15 13:10:41.891545 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:41.891512    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:41.891815 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:41.891796    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:41.892563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:41.892545    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:10:42.757663 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:42.757632    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a70eb796-d33a-443e-83b1-5b305f145179" path="/var/lib/kubelet/pods/a70eb796-d33a-443e-83b1-5b305f145179/volumes"
Sep 15 13:10:45.588136 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:45.588106    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:10:53.695552 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:10:53.695515    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:11:01.545221 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:01.545183    2562 patch_prober.go:28] interesting pod/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 15 13:11:01.545698 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:01.545249    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 15 13:11:03.729855 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:03.729824    2562 generic.go:356] "Generic (PLEG): container finished" podID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerID="e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe" exitCode=0
Sep 15 13:11:03.730231 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:03.729901    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" event={"ID":"58bbef12-2135-43f6-90e3-71cf9f6a538f","Type":"ContainerDied","Data":"e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe"}
Sep 15 13:11:03.730510 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:03.730493    2562 scope.go:117] "RemoveContainer" containerID="e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe"
Sep 15 13:11:03.731844 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:03.731808    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:11:03.731948 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:03.731890    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:11:03.731948 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:03.731939    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:11:04.737821 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:04.737787    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" event={"ID":"58bbef12-2135-43f6-90e3-71cf9f6a538f","Type":"ContainerStarted","Data":"318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b"}
Sep 15 13:11:05.316287 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:05.316246    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:11:05.316460 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:05.316444    2562 patch_prober.go:28] interesting pod/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 13:11:05.316510 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:05.316490    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 13:11:06.316355 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:06.316324    2562 patch_prober.go:28] interesting pod/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 13:11:06.316734 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:06.316389    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 13:11:07.317044 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:07.316999    2562 patch_prober.go:28] interesting pod/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 13:11:07.317499 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:07.317071    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 13:11:08.316614 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:08.316577    2562 patch_prober.go:28] interesting pod/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 13:11:08.316821 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:08.316636    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 13:11:09.316396 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:09.316363    2562 patch_prober.go:28] interesting pod/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 13:11:09.316763 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:09.316415    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 13:11:10.316519 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.316487    2562 patch_prober.go:28] interesting pod/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 13:11:10.316904 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.316547    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 13:11:10.696133 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.696097    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"]
Sep 15 13:11:10.700351 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.700325    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.704169 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.704144    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7e508e61\"/\"kube-root-ca.crt\""
Sep 15 13:11:10.704310 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.704144    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7e508e61\"/\"openshift-service-ca.crt\""
Sep 15 13:11:10.704310 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.704233    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7e508e61\"/\"e2e-lifecycle-model-7e508e61-kserve-self-signed-certs\""
Sep 15 13:11:10.704467 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.704312    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7e508e61\"/\"default-dockercfg-zckhd\""
Sep 15 13:11:10.710016 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.709994    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"]
Sep 15 13:11:10.763605 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.763572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-home\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.763801 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.763633    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-dshm\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.763801 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.763690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtrlg\" (UniqueName: \"kubernetes.io/projected/bf83370d-4442-442f-82a1-fa013ef3e761-kube-api-access-jtrlg\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.763801 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.763790    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf83370d-4442-442f-82a1-fa013ef3e761-tls-certs\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.763978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.763831    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-tmp-dir\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.763978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.763896    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-model-cache\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.864935 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.864896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf83370d-4442-442f-82a1-fa013ef3e761-tls-certs\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.865286 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.865264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-tmp-dir\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.865793 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.865731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-tmp-dir\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.866296 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.865966    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-model-cache\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.866500 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.866485    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-home\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.866897 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.866881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-dshm\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.867402 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.867382    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtrlg\" (UniqueName: \"kubernetes.io/projected/bf83370d-4442-442f-82a1-fa013ef3e761-kube-api-access-jtrlg\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.867948 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.866235    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-model-cache\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.868106 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.866812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-home\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.874891 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.874326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-dshm\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.874891 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.874843    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf83370d-4442-442f-82a1-fa013ef3e761-tls-certs\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:10.877189 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:10.877154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtrlg\" (UniqueName: \"kubernetes.io/projected/bf83370d-4442-442f-82a1-fa013ef3e761-kube-api-access-jtrlg\") pod \"e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") " pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:11.011394 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:11.011316    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:11.288361 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:11.288334    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"]
Sep 15 13:11:11.289891 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:11:11.289864    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf83370d_4442_442f_82a1_fa013ef3e761.slice/crio-7a20efbe95d1aef15de7938a3e16d5b69477d082c0de254c12a62d384ac8dcc0 WatchSource:0}: Error finding container 7a20efbe95d1aef15de7938a3e16d5b69477d082c0de254c12a62d384ac8dcc0: Status 404 returned error can't find the container with id 7a20efbe95d1aef15de7938a3e16d5b69477d082c0de254c12a62d384ac8dcc0
Sep 15 13:11:11.320422 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:11.320394    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:11:11.320780 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:11.320758    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:11:11.321609 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:11.321588    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:11:11.788206 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:11.788160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj" event={"ID":"bf83370d-4442-442f-82a1-fa013ef3e761","Type":"ContainerStarted","Data":"bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21"}
Sep 15 13:11:11.788381 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:11.788211    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj" event={"ID":"bf83370d-4442-442f-82a1-fa013ef3e761","Type":"ContainerStarted","Data":"7a20efbe95d1aef15de7938a3e16d5b69477d082c0de254c12a62d384ac8dcc0"}
Sep 15 13:11:11.809585 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:11.809538    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj" podStartSLOduration=1.809522155 podStartE2EDuration="1.809522155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:11:11.806954114 +0000 UTC m=+1589.599655846" watchObservedRunningTime="2026-09-15 13:11:11.809522155 +0000 UTC m=+1589.602223876"
Sep 15 13:11:18.311705 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:18.311667    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:11:21.011929 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:21.011898    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:21.012321 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:21.011963    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:21.024575 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:21.024541    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:21.857811 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:21.857775    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:11:43.022277 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:11:43.022184    2562 scope.go:117] "RemoveContainer" containerID="d2aa677382c9c804005c62f078f27fd11ae8d82eea9623b0a495d79a6e6197dc"
Sep 15 13:12:05.204628 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:05.204591    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"]
Sep 15 13:12:20.225074 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.224986    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx" podUID="212d55d6-91d9-456d-ae90-e6d0a61a08bb" containerName="main" containerID="cri-o://e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5" gracePeriod=45
Sep 15 13:12:20.484841 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.484766    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:12:20.641861 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.641826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-model-cache\") pod \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") "
Sep 15 13:12:20.642071 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.641888    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-dshm\") pod \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") "
Sep 15 13:12:20.642071 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.641914    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsn4f\" (UniqueName: \"kubernetes.io/projected/212d55d6-91d9-456d-ae90-e6d0a61a08bb-kube-api-access-nsn4f\") pod \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") "
Sep 15 13:12:20.642071 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.641947    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tmp-dir\") pod \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") "
Sep 15 13:12:20.642071 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.642061    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tls-certs\") pod \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") "
Sep 15 13:12:20.642308 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.642131    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-home\") pod \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\" (UID: \"212d55d6-91d9-456d-ae90-e6d0a61a08bb\") "
Sep 15 13:12:20.642308 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.642130    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "212d55d6-91d9-456d-ae90-e6d0a61a08bb" (UID: "212d55d6-91d9-456d-ae90-e6d0a61a08bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:12:20.642308 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.642272    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "212d55d6-91d9-456d-ae90-e6d0a61a08bb" (UID: "212d55d6-91d9-456d-ae90-e6d0a61a08bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:12:20.642466 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.642433    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-home" (OuterVolumeSpecName: "home") pod "212d55d6-91d9-456d-ae90-e6d0a61a08bb" (UID: "212d55d6-91d9-456d-ae90-e6d0a61a08bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:12:20.642577 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.642558    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-model-cache\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:12:20.642645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.642583    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tmp-dir\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:12:20.642645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.642596    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-home\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:12:20.644288 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.644262    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "212d55d6-91d9-456d-ae90-e6d0a61a08bb" (UID: "212d55d6-91d9-456d-ae90-e6d0a61a08bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:12:20.644398 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.644301    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/212d55d6-91d9-456d-ae90-e6d0a61a08bb-kube-api-access-nsn4f" (OuterVolumeSpecName: "kube-api-access-nsn4f") pod "212d55d6-91d9-456d-ae90-e6d0a61a08bb" (UID: "212d55d6-91d9-456d-ae90-e6d0a61a08bb"). InnerVolumeSpecName "kube-api-access-nsn4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:12:20.644398 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.644352    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-dshm" (OuterVolumeSpecName: "dshm") pod "212d55d6-91d9-456d-ae90-e6d0a61a08bb" (UID: "212d55d6-91d9-456d-ae90-e6d0a61a08bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:12:20.743627 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.743540    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/212d55d6-91d9-456d-ae90-e6d0a61a08bb-dshm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:12:20.743627 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.743569    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsn4f\" (UniqueName: \"kubernetes.io/projected/212d55d6-91d9-456d-ae90-e6d0a61a08bb-kube-api-access-nsn4f\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:12:20.743627 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:20.743580    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212d55d6-91d9-456d-ae90-e6d0a61a08bb-tls-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:12:21.194973 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.194939    2562 generic.go:356] "Generic (PLEG): container finished" podID="212d55d6-91d9-456d-ae90-e6d0a61a08bb" containerID="e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5" exitCode=0
Sep 15 13:12:21.195182 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.194977    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"
Sep 15 13:12:21.195182 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.195006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx" event={"ID":"212d55d6-91d9-456d-ae90-e6d0a61a08bb","Type":"ContainerDied","Data":"e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5"}
Sep 15 13:12:21.195182 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.195069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx" event={"ID":"212d55d6-91d9-456d-ae90-e6d0a61a08bb","Type":"ContainerDied","Data":"45ed707ebdfc9dafd4991ee0de3a6caf788c972fa5d69912243b0cc0a980e744"}
Sep 15 13:12:21.195182 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.195091    2562 scope.go:117] "RemoveContainer" containerID="e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5"
Sep 15 13:12:21.204230 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.204210    2562 scope.go:117] "RemoveContainer" containerID="e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5"
Sep 15 13:12:21.204480 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:12:21.204463    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5\": container with ID starting with e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5 not found: ID does not exist" containerID="e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5"
Sep 15 13:12:21.204551 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.204487    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5"} err="failed to get container status \"e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5\": rpc error: code = NotFound desc = could not find container \"e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5\": container with ID starting with e2c0c48772275ede0a9b0d25fa5388801f747d97b632cacb289aff5013f4aaa5 not found: ID does not exist"
Sep 15 13:12:21.214797 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.214771    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"]
Sep 15 13:12:21.219307 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:21.219286    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-6c9f0b/premium-w0-6c9f0b-kserve-9d545964b-l8ptx"]
Sep 15 13:12:22.756800 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:22.756769    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="212d55d6-91d9-456d-ae90-e6d0a61a08bb" path="/var/lib/kubelet/pods/212d55d6-91d9-456d-ae90-e6d0a61a08bb/volumes"
Sep 15 13:12:49.483623 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:12:49.483588    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:14:14.280530 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:14.280495    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:14:39.276793 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:39.276708    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"]
Sep 15 13:14:39.277613 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:39.277561    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy" containerID="cri-o://df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de" gracePeriod=30
Sep 15 13:14:40.110172 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.110130    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"]
Sep 15 13:14:40.110885 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.110855    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="212d55d6-91d9-456d-ae90-e6d0a61a08bb" containerName="main"
Sep 15 13:14:40.110885 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.110879    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="212d55d6-91d9-456d-ae90-e6d0a61a08bb" containerName="main"
Sep 15 13:14:40.111061 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.111019    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="212d55d6-91d9-456d-ae90-e6d0a61a08bb" containerName="main"
Sep 15 13:14:40.114734 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.114714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.118618 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.118600    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-aa8ef1-openshift-default-dockercfg-4vgsk\""
Sep 15 13:14:40.130659 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.130627    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"]
Sep 15 13:14:40.266313 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266267    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-credential-socket\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.266313 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-data\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.266535 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266376    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2hr5\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-kube-api-access-k2hr5\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.266535 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72c243a2-0019-4f53-b591-53708d8cdef6-istio-podinfo\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.266535 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-certs\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.266638 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-socket\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.266638 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-istio-token\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.266638 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266615    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72c243a2-0019-4f53-b591-53708d8cdef6-istiod-ca-cert\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.266740 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.266661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-envoy\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.367951 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.367854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-data\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.367951 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.367908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2hr5\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-kube-api-access-k2hr5\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.367980    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72c243a2-0019-4f53-b591-53708d8cdef6-istio-podinfo\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-certs\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-socket\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-istio-token\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72c243a2-0019-4f53-b591-53708d8cdef6-istiod-ca-cert\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368344    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-data\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-envoy\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-certs\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368517    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-socket\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368527    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-credential-socket\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.368921 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.368838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-credential-socket\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.369065 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.369018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72c243a2-0019-4f53-b591-53708d8cdef6-istiod-ca-cert\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.370488 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.370467    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-envoy\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.370569 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.370552    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72c243a2-0019-4f53-b591-53708d8cdef6-istio-podinfo\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.376719 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.376693    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-istio-token\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.376996 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.376974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2hr5\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-kube-api-access-k2hr5\") pod \"e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") " pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.427976 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.427933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:40.713253 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.713225    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"]
Sep 15 13:14:40.715075 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:14:40.715014    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c243a2_0019_4f53_b591_53708d8cdef6.slice/crio-ddd24570c130ccb8aa8260d19d9fce49855811da706bc91a222dd9c95d6cb339 WatchSource:0}: Error finding container ddd24570c130ccb8aa8260d19d9fce49855811da706bc91a222dd9c95d6cb339: Status 404 returned error can't find the container with id ddd24570c130ccb8aa8260d19d9fce49855811da706bc91a222dd9c95d6cb339
Sep 15 13:14:40.717762 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.717719    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:14:40.717844 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.717816    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:14:40.717884 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:40.717866    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:14:41.017083 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:41.017021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" event={"ID":"72c243a2-0019-4f53-b591-53708d8cdef6","Type":"ContainerStarted","Data":"409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b"}
Sep 15 13:14:41.017258 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:41.017111    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" event={"ID":"72c243a2-0019-4f53-b591-53708d8cdef6","Type":"ContainerStarted","Data":"ddd24570c130ccb8aa8260d19d9fce49855811da706bc91a222dd9c95d6cb339"}
Sep 15 13:14:41.041694 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:41.041632    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podStartSLOduration=1.041610311 podStartE2EDuration="1.041610311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:14:41.039315254 +0000 UTC m=+1798.832016986" watchObservedRunningTime="2026-09-15 13:14:41.041610311 +0000 UTC m=+1798.834312046"
Sep 15 13:14:41.429186 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:41.429150    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:41.434406 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:41.434379    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:42.024394 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:42.024364    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:42.025490 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:42.025463    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:14:42.859966 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:42.859937    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:14:42.863977 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:42.863955    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:14:42.865890 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:42.865868    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:14:42.870642 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:42.870621    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:14:44.541437 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.541369    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 13:14:44.717828 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.717796    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-envoy\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.717828 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.717829    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-data\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.718141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.717862    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-podinfo\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.718141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.717905    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-socket\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.718141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.717966    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vchsr\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-kube-api-access-vchsr\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.718141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718098    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-credential-socket\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.718359 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718150    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-certs\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.718359 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718188    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-token\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.718359 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718234    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:14:44.718359 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718260    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istiod-ca-cert\") pod \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\" (UID: \"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0\") "
Sep 15 13:14:44.718563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718347    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:14:44.718563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718400    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:14:44.718563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718492    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:14:44.718761 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718728    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:14:44.718826 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718776    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:44.718826 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718798    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:44.718826 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718812    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:44.718931 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.718828    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:44.720332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.720289    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:14:44.720332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.720313    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-kube-api-access-vchsr" (OuterVolumeSpecName: "kube-api-access-vchsr") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "kube-api-access-vchsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:14:44.720620 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.720601    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:14:44.720694 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.720654    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" (UID: "a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:14:44.820268 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.820189    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:44.820268 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.820222    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:44.820268 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.820235    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:44.820268 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.820248    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:44.820268 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:44.820263    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vchsr\" (UniqueName: \"kubernetes.io/projected/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0-kube-api-access-vchsr\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:14:45.059996 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.059965    2562 generic.go:356] "Generic (PLEG): container finished" podID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerID="df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de" exitCode=0
Sep 15 13:14:45.060180 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.060018    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"
Sep 15 13:14:45.060180 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.060067    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" event={"ID":"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0","Type":"ContainerDied","Data":"df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de"}
Sep 15 13:14:45.060180 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.060133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs" event={"ID":"a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0","Type":"ContainerDied","Data":"cd72a715ede33d5fb1996f7cd8337d220cd70c7264986ea2a8da42091f64f155"}
Sep 15 13:14:45.060180 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.060149    2562 scope.go:117] "RemoveContainer" containerID="df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de"
Sep 15 13:14:45.069380 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.069362    2562 scope.go:117] "RemoveContainer" containerID="7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68"
Sep 15 13:14:45.077527 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.077510    2562 scope.go:117] "RemoveContainer" containerID="df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de"
Sep 15 13:14:45.077788 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:14:45.077771    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de\": container with ID starting with df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de not found: ID does not exist" containerID="df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de"
Sep 15 13:14:45.077836 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.077794    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de"} err="failed to get container status \"df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de\": rpc error: code = NotFound desc = could not find container \"df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de\": container with ID starting with df58d291d784bb6d927601aa2a25509b8d90f215701a811405cf0cfdee7ce8de not found: ID does not exist"
Sep 15 13:14:45.077836 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.077811    2562 scope.go:117] "RemoveContainer" containerID="7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68"
Sep 15 13:14:45.078082 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:14:45.078066    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68\": container with ID starting with 7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68 not found: ID does not exist" containerID="7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68"
Sep 15 13:14:45.078136 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.078089    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68"} err="failed to get container status \"7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68\": rpc error: code = NotFound desc = could not find container \"7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68\": container with ID starting with 7e9c74af187f0b399aad590c3b25bd673c1c5d6d89277ee952238ff8b00a3f68 not found: ID does not exist"
Sep 15 13:14:45.099126 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.099097    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"]
Sep 15 13:14:45.103260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:45.103237    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1c35dd-openshift-default-7775fb6db4-h7qxs"]
Sep 15 13:14:46.349469 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.349438    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:14:46.756624 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.756589    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" path="/var/lib/kubelet/pods/a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0/volumes"
Sep 15 13:14:46.786791 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.786759    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"]
Sep 15 13:14:46.787228 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.787215    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy"
Sep 15 13:14:46.787287 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.787230    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy"
Sep 15 13:14:46.787326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.787317    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy"
Sep 15 13:14:46.787367 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.787328    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy"
Sep 15 13:14:46.790811 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.790790    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:46.793934 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.793913    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-aa8ef1-serving-cert\""
Sep 15 13:14:46.801777 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.801746    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"]
Sep 15 13:14:46.944860 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.944822    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8gxw\" (UniqueName: \"kubernetes.io/projected/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-kube-api-access-t8gxw\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:46.945048 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.944865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:46.945110 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.945019    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:46.945148 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:46.945132    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.045914 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.045808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.045914 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.045875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.045914 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.045914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8gxw\" (UniqueName: \"kubernetes.io/projected/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-kube-api-access-t8gxw\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.046268 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.045935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.046268 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:14:47.045990    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-serving-cert: secret "maas-api-e2e-shared-a-w3-aa8ef1-serving-cert" not found
Sep 15 13:14:47.046268 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:14:47.046114    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls podName:55f5a4d5-a122-49e6-85f1-f5aa2ce1508c nodeName:}" failed. No retries permitted until 2026-09-15 13:14:47.546092351 +0000 UTC m=+1805.338794074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls") pod "maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd" (UID: "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c") : secret "maas-api-e2e-shared-a-w3-aa8ef1-serving-cert" not found
Sep 15 13:14:47.046587 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.046569    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.048534 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.048508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.073440 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.073409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8gxw\" (UniqueName: \"kubernetes.io/projected/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-kube-api-access-t8gxw\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.552556 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.552518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.555278 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.555252    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:47.703448 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:47.703405    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:48.002249 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:48.002222    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"]
Sep 15 13:14:48.004506 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:14:48.004475    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f5a4d5_a122_49e6_85f1_f5aa2ce1508c.slice/crio-a964cbe9beefe5689ff2183f69cc093bf05bf9adbf90e678b64ea8695040fb47 WatchSource:0}: Error finding container a964cbe9beefe5689ff2183f69cc093bf05bf9adbf90e678b64ea8695040fb47: Status 404 returned error can't find the container with id a964cbe9beefe5689ff2183f69cc093bf05bf9adbf90e678b64ea8695040fb47
Sep 15 13:14:48.082504 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:48.082467    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd" event={"ID":"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c","Type":"ContainerStarted","Data":"a964cbe9beefe5689ff2183f69cc093bf05bf9adbf90e678b64ea8695040fb47"}
Sep 15 13:14:49.094122 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:49.093256    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd" event={"ID":"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c","Type":"ContainerStarted","Data":"4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b"}
Sep 15 13:14:49.094122 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:49.093681    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:14:49.119542 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:49.119496    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd" podStartSLOduration=2.585075325 podStartE2EDuration="3.119481566s" totalImagesPullingTime="534.406241ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:14:49.117917068 +0000 UTC m=+1806.910618798" watchObservedRunningTime="2026-09-15 13:14:49.119481566 +0000 UTC m=+1806.912183296"
Sep 15 13:14:55.104212 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:14:55.104182    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:15:00.146420 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.146371    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"]
Sep 15 13:15:00.147179 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.147153    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy"
Sep 15 13:15:00.147280 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.147182    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a96fa1d9-5b27-4289-9913-bf4dbb7fe9d0" containerName="istio-proxy"
Sep 15 13:15:00.152596 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.152566    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"
Sep 15 13:15:00.156902 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.156877    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:15:00.167866 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.167839    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"]
Sep 15 13:15:00.172359 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.172273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"
Sep 15 13:15:00.172359 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.172339    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"]
Sep 15 13:15:00.179310 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.179283    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"]
Sep 15 13:15:00.184623 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.184598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"
Sep 15 13:15:00.185232 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.185204    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"]
Sep 15 13:15:00.199371 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.198549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjpqf\" (UniqueName: \"kubernetes.io/projected/d392cfa0-2845-419a-81dc-6f9d8c799f6d-kube-api-access-sjpqf\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m\" (UID: \"d392cfa0-2845-419a-81dc-6f9d8c799f6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"
Sep 15 13:15:00.204658 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.204556    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"]
Sep 15 13:15:00.299934 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.299897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nhx6\" (UniqueName: \"kubernetes.io/projected/aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092-kube-api-access-2nhx6\") pod \"maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b\" (UID: \"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"
Sep 15 13:15:00.300161 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.300001    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjpqf\" (UniqueName: \"kubernetes.io/projected/d392cfa0-2845-419a-81dc-6f9d8c799f6d-kube-api-access-sjpqf\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m\" (UID: \"d392cfa0-2845-419a-81dc-6f9d8c799f6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"
Sep 15 13:15:00.300240 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.300167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwdhc\" (UniqueName: \"kubernetes.io/projected/84923d91-5727-4fe7-9f4b-cfba64754e4d-kube-api-access-zwdhc\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6\" (UID: \"84923d91-5727-4fe7-9f4b-cfba64754e4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"
Sep 15 13:15:00.309333 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.309303    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjpqf\" (UniqueName: \"kubernetes.io/projected/d392cfa0-2845-419a-81dc-6f9d8c799f6d-kube-api-access-sjpqf\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m\" (UID: \"d392cfa0-2845-419a-81dc-6f9d8c799f6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"
Sep 15 13:15:00.401445 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.401360    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwdhc\" (UniqueName: \"kubernetes.io/projected/84923d91-5727-4fe7-9f4b-cfba64754e4d-kube-api-access-zwdhc\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6\" (UID: \"84923d91-5727-4fe7-9f4b-cfba64754e4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"
Sep 15 13:15:00.401445 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.401440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nhx6\" (UniqueName: \"kubernetes.io/projected/aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092-kube-api-access-2nhx6\") pod \"maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b\" (UID: \"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"
Sep 15 13:15:00.410173 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.410142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwdhc\" (UniqueName: \"kubernetes.io/projected/84923d91-5727-4fe7-9f4b-cfba64754e4d-kube-api-access-zwdhc\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6\" (UID: \"84923d91-5727-4fe7-9f4b-cfba64754e4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"
Sep 15 13:15:00.410295 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.410281    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nhx6\" (UniqueName: \"kubernetes.io/projected/aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092-kube-api-access-2nhx6\") pod \"maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b\" (UID: \"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"
Sep 15 13:15:00.467775 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.467740    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"
Sep 15 13:15:00.487950 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.487924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"
Sep 15 13:15:00.496097 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.496063    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"
Sep 15 13:15:00.833123 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.833086    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"]
Sep 15 13:15:00.843521 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.841058    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"]
Sep 15 13:15:00.849080 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:15:00.844148    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84923d91_5727_4fe7_9f4b_cfba64754e4d.slice/crio-fc6b816090451f8f14858eb5d0a24b70a927a989164965eee94f08ea1b0cbfda WatchSource:0}: Error finding container fc6b816090451f8f14858eb5d0a24b70a927a989164965eee94f08ea1b0cbfda: Status 404 returned error can't find the container with id fc6b816090451f8f14858eb5d0a24b70a927a989164965eee94f08ea1b0cbfda
Sep 15 13:15:00.854977 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:00.854823    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"]
Sep 15 13:15:01.174597 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:01.174348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6" event={"ID":"84923d91-5727-4fe7-9f4b-cfba64754e4d","Type":"ContainerStarted","Data":"06e94205d8f2188b2b75b8441cd638b0e4de4ab9598d606b5772a581a606e01f"}
Sep 15 13:15:01.174597 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:01.174420    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6" event={"ID":"84923d91-5727-4fe7-9f4b-cfba64754e4d","Type":"ContainerStarted","Data":"fc6b816090451f8f14858eb5d0a24b70a927a989164965eee94f08ea1b0cbfda"}
Sep 15 13:15:01.179243 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:01.179160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m" event={"ID":"d392cfa0-2845-419a-81dc-6f9d8c799f6d","Type":"ContainerStarted","Data":"9536a6c591c2c72ff6d8a87f6f676857d12e188d0639bea233260de36483ed6f"}
Sep 15 13:15:01.180614 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:01.180588    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b" event={"ID":"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092","Type":"ContainerStarted","Data":"50e2959d9e96515219d86a4ab00f823209d964a28ea59bff95674c7508583061"}
Sep 15 13:15:01.193666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:01.193622    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6" podStartSLOduration=1.193609268 podStartE2EDuration="1.193609268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:15:01.191198317 +0000 UTC m=+1818.983900047" watchObservedRunningTime="2026-09-15 13:15:01.193609268 +0000 UTC m=+1818.986311022"
Sep 15 13:15:02.188188 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:02.188144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b" event={"ID":"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092","Type":"ContainerStarted","Data":"634f37000922d198f32e220d3b80ab4d2994bb4d79712462fe14717e8d5544f4"}
Sep 15 13:15:02.189922 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:02.189888    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m" event={"ID":"d392cfa0-2845-419a-81dc-6f9d8c799f6d","Type":"ContainerStarted","Data":"1b0e4731ba9bfcb2ac1a8662f550cb4f8ae877ba65ee53db4e954f61e4d55c0c"}
Sep 15 13:15:02.208770 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:02.208725    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b" podStartSLOduration=2.208712334 podStartE2EDuration="2.208712334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:15:02.20620045 +0000 UTC m=+1819.998902181" watchObservedRunningTime="2026-09-15 13:15:02.208712334 +0000 UTC m=+1820.001414064"
Sep 15 13:15:02.226485 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:02.226437    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m" podStartSLOduration=2.226417313 podStartE2EDuration="2.226417313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:15:02.223467715 +0000 UTC m=+1820.016169446" watchObservedRunningTime="2026-09-15 13:15:02.226417313 +0000 UTC m=+1820.019119044"
Sep 15 13:15:03.197291 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:03.197254    2562 generic.go:356] "Generic (PLEG): container finished" podID="84923d91-5727-4fe7-9f4b-cfba64754e4d" containerID="06e94205d8f2188b2b75b8441cd638b0e4de4ab9598d606b5772a581a606e01f" exitCode=0
Sep 15 13:15:03.197738 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:03.197327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6" event={"ID":"84923d91-5727-4fe7-9f4b-cfba64754e4d","Type":"ContainerDied","Data":"06e94205d8f2188b2b75b8441cd638b0e4de4ab9598d606b5772a581a606e01f"}
Sep 15 13:15:03.198938 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:03.198912    2562 generic.go:356] "Generic (PLEG): container finished" podID="d392cfa0-2845-419a-81dc-6f9d8c799f6d" containerID="1b0e4731ba9bfcb2ac1a8662f550cb4f8ae877ba65ee53db4e954f61e4d55c0c" exitCode=0
Sep 15 13:15:03.199082 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:03.198965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m" event={"ID":"d392cfa0-2845-419a-81dc-6f9d8c799f6d","Type":"ContainerDied","Data":"1b0e4731ba9bfcb2ac1a8662f550cb4f8ae877ba65ee53db4e954f61e4d55c0c"}
Sep 15 13:15:03.200436 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:03.200417    2562 generic.go:356] "Generic (PLEG): container finished" podID="aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092" containerID="634f37000922d198f32e220d3b80ab4d2994bb4d79712462fe14717e8d5544f4" exitCode=0
Sep 15 13:15:03.200510 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:03.200451    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b" event={"ID":"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092","Type":"ContainerDied","Data":"634f37000922d198f32e220d3b80ab4d2994bb4d79712462fe14717e8d5544f4"}
Sep 15 13:15:04.043279 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.043243    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"]
Sep 15 13:15:04.043804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.043755    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v" podUID="28e25595-9bc1-4925-866b-f545b2817819" containerName="maas-api" containerID="cri-o://ffbc122e7ff8f7a16163fd5495a4d28e86c06e8478fb793b844b56748f33ebf2" gracePeriod=30
Sep 15 13:15:04.213561 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.213525    2562 generic.go:356] "Generic (PLEG): container finished" podID="28e25595-9bc1-4925-866b-f545b2817819" containerID="ffbc122e7ff8f7a16163fd5495a4d28e86c06e8478fb793b844b56748f33ebf2" exitCode=0
Sep 15 13:15:04.214008 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.213774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v" event={"ID":"28e25595-9bc1-4925-866b-f545b2817819","Type":"ContainerDied","Data":"ffbc122e7ff8f7a16163fd5495a4d28e86c06e8478fb793b844b56748f33ebf2"}
Sep 15 13:15:04.364336 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.362096    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:15:04.455199 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.455163    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e25595-9bc1-4925-866b-f545b2817819-openshift-service-ca\") pod \"28e25595-9bc1-4925-866b-f545b2817819\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") "
Sep 15 13:15:04.455339 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.455220    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-metrics-certs\") pod \"28e25595-9bc1-4925-866b-f545b2817819\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") "
Sep 15 13:15:04.455339 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.455315    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfgxs\" (UniqueName: \"kubernetes.io/projected/28e25595-9bc1-4925-866b-f545b2817819-kube-api-access-hfgxs\") pod \"28e25595-9bc1-4925-866b-f545b2817819\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") "
Sep 15 13:15:04.455413 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.455355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls\") pod \"28e25595-9bc1-4925-866b-f545b2817819\" (UID: \"28e25595-9bc1-4925-866b-f545b2817819\") "
Sep 15 13:15:04.456706 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.456609    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e25595-9bc1-4925-866b-f545b2817819-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28e25595-9bc1-4925-866b-f545b2817819" (UID: "28e25595-9bc1-4925-866b-f545b2817819"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:15:04.462202 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.462156    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28e25595-9bc1-4925-866b-f545b2817819" (UID: "28e25595-9bc1-4925-866b-f545b2817819"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:15:04.462317 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.462254    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e25595-9bc1-4925-866b-f545b2817819-kube-api-access-hfgxs" (OuterVolumeSpecName: "kube-api-access-hfgxs") pod "28e25595-9bc1-4925-866b-f545b2817819" (UID: "28e25595-9bc1-4925-866b-f545b2817819"). InnerVolumeSpecName "kube-api-access-hfgxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:15:04.462317 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.462278    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28e25595-9bc1-4925-866b-f545b2817819" (UID: "28e25595-9bc1-4925-866b-f545b2817819"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:15:04.472728 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.472703    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"
Sep 15 13:15:04.492162 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.492131    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"
Sep 15 13:15:04.502516 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.502426    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"
Sep 15 13:15:04.556805 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.556720    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjpqf\" (UniqueName: \"kubernetes.io/projected/d392cfa0-2845-419a-81dc-6f9d8c799f6d-kube-api-access-sjpqf\") pod \"d392cfa0-2845-419a-81dc-6f9d8c799f6d\" (UID: \"d392cfa0-2845-419a-81dc-6f9d8c799f6d\") "
Sep 15 13:15:04.556980 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.556881    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nhx6\" (UniqueName: \"kubernetes.io/projected/aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092-kube-api-access-2nhx6\") pod \"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092\" (UID: \"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092\") "
Sep 15 13:15:04.557335 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.557313    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfgxs\" (UniqueName: \"kubernetes.io/projected/28e25595-9bc1-4925-866b-f545b2817819-kube-api-access-hfgxs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:04.557446 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.557337    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:04.557446 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.557352    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e25595-9bc1-4925-866b-f545b2817819-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:04.557446 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.557360    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e25595-9bc1-4925-866b-f545b2817819-metrics-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:04.559774 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.559741    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d392cfa0-2845-419a-81dc-6f9d8c799f6d-kube-api-access-sjpqf" (OuterVolumeSpecName: "kube-api-access-sjpqf") pod "d392cfa0-2845-419a-81dc-6f9d8c799f6d" (UID: "d392cfa0-2845-419a-81dc-6f9d8c799f6d"). InnerVolumeSpecName "kube-api-access-sjpqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:15:04.560369 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.560340    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092-kube-api-access-2nhx6" (OuterVolumeSpecName: "kube-api-access-2nhx6") pod "aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092" (UID: "aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092"). InnerVolumeSpecName "kube-api-access-2nhx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:15:04.658341 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.658307    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwdhc\" (UniqueName: \"kubernetes.io/projected/84923d91-5727-4fe7-9f4b-cfba64754e4d-kube-api-access-zwdhc\") pod \"84923d91-5727-4fe7-9f4b-cfba64754e4d\" (UID: \"84923d91-5727-4fe7-9f4b-cfba64754e4d\") "
Sep 15 13:15:04.658602 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.658588    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nhx6\" (UniqueName: \"kubernetes.io/projected/aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092-kube-api-access-2nhx6\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:04.658656 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.658603    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjpqf\" (UniqueName: \"kubernetes.io/projected/d392cfa0-2845-419a-81dc-6f9d8c799f6d-kube-api-access-sjpqf\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:04.660360 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.660325    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84923d91-5727-4fe7-9f4b-cfba64754e4d-kube-api-access-zwdhc" (OuterVolumeSpecName: "kube-api-access-zwdhc") pod "84923d91-5727-4fe7-9f4b-cfba64754e4d" (UID: "84923d91-5727-4fe7-9f4b-cfba64754e4d"). InnerVolumeSpecName "kube-api-access-zwdhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:15:04.767304 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:04.767271    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwdhc\" (UniqueName: \"kubernetes.io/projected/84923d91-5727-4fe7-9f4b-cfba64754e4d-kube-api-access-zwdhc\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:05.223007 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.222456    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b"
Sep 15 13:15:05.223007 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.222464    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b" event={"ID":"aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092","Type":"ContainerDied","Data":"50e2959d9e96515219d86a4ab00f823209d964a28ea59bff95674c7508583061"}
Sep 15 13:15:05.223007 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.222536    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50e2959d9e96515219d86a4ab00f823209d964a28ea59bff95674c7508583061"
Sep 15 13:15:05.225142 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.225109    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6" event={"ID":"84923d91-5727-4fe7-9f4b-cfba64754e4d","Type":"ContainerDied","Data":"fc6b816090451f8f14858eb5d0a24b70a927a989164965eee94f08ea1b0cbfda"}
Sep 15 13:15:05.225279 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.225156    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc6b816090451f8f14858eb5d0a24b70a927a989164965eee94f08ea1b0cbfda"
Sep 15 13:15:05.225279 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.225219    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"
Sep 15 13:15:05.227355 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.227337    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"
Sep 15 13:15:05.227482 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.227361    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v" event={"ID":"28e25595-9bc1-4925-866b-f545b2817819","Type":"ContainerDied","Data":"c2c2db70a704773201709bff1487191fc35c79437f96ff1c9fffc391cfff9b5a"}
Sep 15 13:15:05.227482 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.227408    2562 scope.go:117] "RemoveContainer" containerID="ffbc122e7ff8f7a16163fd5495a4d28e86c06e8478fb793b844b56748f33ebf2"
Sep 15 13:15:05.229349 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.229314    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m" event={"ID":"d392cfa0-2845-419a-81dc-6f9d8c799f6d","Type":"ContainerDied","Data":"9536a6c591c2c72ff6d8a87f6f676857d12e188d0639bea233260de36483ed6f"}
Sep 15 13:15:05.229450 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.229352    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9536a6c591c2c72ff6d8a87f6f676857d12e188d0639bea233260de36483ed6f"
Sep 15 13:15:05.229450 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.229367    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"
Sep 15 13:15:05.264472 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.264439    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"]
Sep 15 13:15:05.272660 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.272636    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e508e61-67cb455cc8-kwm4v"]
Sep 15 13:15:05.798802 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:05.798767    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:15:06.756676 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:06.756645    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e25595-9bc1-4925-866b-f545b2817819" path="/var/lib/kubelet/pods/28e25595-9bc1-4925-866b-f545b2817819/volumes"
Sep 15 13:15:07.245059 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:07.244986    2562 generic.go:356] "Generic (PLEG): container finished" podID="72c243a2-0019-4f53-b591-53708d8cdef6" containerID="409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b" exitCode=0
Sep 15 13:15:07.245266 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:07.245135    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" event={"ID":"72c243a2-0019-4f53-b591-53708d8cdef6","Type":"ContainerDied","Data":"409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b"}
Sep 15 13:15:07.245619 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:07.245601    2562 scope.go:117] "RemoveContainer" containerID="409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b"
Sep 15 13:15:07.246828 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:07.246801    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:07.246897 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:07.246882    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:07.246936 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:07.246920    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:08.254238 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.254206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" event={"ID":"72c243a2-0019-4f53-b591-53708d8cdef6","Type":"ContainerStarted","Data":"de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d"}
Sep 15 13:15:08.430342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.430244    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:15:08.430680 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.430648    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 13:15:08.430790 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.430727    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 13:15:08.529126 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.528817    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"]
Sep 15 13:15:08.542602 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542563    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e25595-9bc1-4925-866b-f545b2817819" containerName="maas-api"
Sep 15 13:15:08.542602 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542600    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e25595-9bc1-4925-866b-f545b2817819" containerName="maas-api"
Sep 15 13:15:08.542842 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542612    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d392cfa0-2845-419a-81dc-6f9d8c799f6d" containerName="cleanup"
Sep 15 13:15:08.542842 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542621    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d392cfa0-2845-419a-81dc-6f9d8c799f6d" containerName="cleanup"
Sep 15 13:15:08.542842 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542641    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092" containerName="cleanup"
Sep 15 13:15:08.542842 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542649    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092" containerName="cleanup"
Sep 15 13:15:08.542842 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542687    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84923d91-5727-4fe7-9f4b-cfba64754e4d" containerName="cleanup"
Sep 15 13:15:08.542842 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542694    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="84923d91-5727-4fe7-9f4b-cfba64754e4d" containerName="cleanup"
Sep 15 13:15:08.543229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542810    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d392cfa0-2845-419a-81dc-6f9d8c799f6d" containerName="cleanup"
Sep 15 13:15:08.543229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542866    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092" containerName="cleanup"
Sep 15 13:15:08.543229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542900    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="84923d91-5727-4fe7-9f4b-cfba64754e4d" containerName="cleanup"
Sep 15 13:15:08.543229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.542913    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e25595-9bc1-4925-866b-f545b2817819" containerName="maas-api"
Sep 15 13:15:08.557058 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.554048    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"]
Sep 15 13:15:08.557058 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.554178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.560654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.560601    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-38d90f-openshift-default-dockercfg-d7gkb\""
Sep 15 13:15:08.712416 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.712386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-socket\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.712682 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.712657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-certs\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.712783 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.712771    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-envoy\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.712888 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.712878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-podinfo\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.713000 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.712990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-credential-socket\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.713128 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.713111    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istiod-ca-cert\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.713210 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.713169    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw7wp\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-kube-api-access-kw7wp\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.713267 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.713234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-data\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.713326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.713302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-token\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.814828 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.814729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-certs\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.814828 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.814764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-envoy\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.814828 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.814805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-podinfo\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.814843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-credential-socket\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.814880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istiod-ca-cert\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.814940    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw7wp\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-kube-api-access-kw7wp\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.814995    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-data\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.815076    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-token\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.815130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-socket\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815469 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.815205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-certs\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815469 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.815234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-credential-socket\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815582 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.815465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-data\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815582 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.815527    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-socket\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.815855 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.815837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istiod-ca-cert\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.817326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.817305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-envoy\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.817546 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.817528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-podinfo\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.826051 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.824793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw7wp\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-kube-api-access-kw7wp\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.826051 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.824901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-token\") pod \"e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") " pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:08.869824 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:08.869788    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:09.169177 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.169146    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"]
Sep 15 13:15:09.170635 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:15:09.170576    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6e1bad_60f7_43a4_b48d_aa4965c057c2.slice/crio-795488e830d8179b3909f2e46558d490703d0ab41eed6b6aaf950dd8e4e5eda3 WatchSource:0}: Error finding container 795488e830d8179b3909f2e46558d490703d0ab41eed6b6aaf950dd8e4e5eda3: Status 404 returned error can't find the container with id 795488e830d8179b3909f2e46558d490703d0ab41eed6b6aaf950dd8e4e5eda3
Sep 15 13:15:09.177052 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.177004    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:09.177156 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.177082    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:09.177156 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.177115    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:09.262585 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.262554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" event={"ID":"3f6e1bad-60f7-43a4-b48d-aa4965c057c2","Type":"ContainerStarted","Data":"f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f"}
Sep 15 13:15:09.263163 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.263117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" event={"ID":"3f6e1bad-60f7-43a4-b48d-aa4965c057c2","Type":"ContainerStarted","Data":"795488e830d8179b3909f2e46558d490703d0ab41eed6b6aaf950dd8e4e5eda3"}
Sep 15 13:15:09.295798 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.295746    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" podStartSLOduration=1.295729984 podStartE2EDuration="1.295729984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:15:09.292580938 +0000 UTC m=+1827.085282695" watchObservedRunningTime="2026-09-15 13:15:09.295729984 +0000 UTC m=+1827.088431714"
Sep 15 13:15:09.428884 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.428851    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 13:15:09.429228 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.429196    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 13:15:09.612049 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.611998    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"]
Sep 15 13:15:09.870197 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:09.870106    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:10.429399 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:10.429357    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 13:15:10.429905 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:10.429431    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 13:15:10.875404 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:10.875333    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:11.276101 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:11.276066    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:11.277152 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:11.277130    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:11.432055 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:11.428509    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 13:15:11.432055 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:11.428584    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 13:15:12.026069 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:12.025162    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:15:12.029913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:12.029887    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"]
Sep 15 13:15:12.034263 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:12.034243    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c35dd-g7zzl"]
Sep 15 13:15:12.429323 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:12.429286    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 13:15:12.429516 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:12.429355    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 13:15:12.764924 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:12.764831    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab5619f5-f511-48fb-98ed-f1b00b5f6f83" path="/var/lib/kubelet/pods/ab5619f5-f511-48fb-98ed-f1b00b5f6f83/volumes"
Sep 15 13:15:13.428990 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:13.428954    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 13:15:13.429323 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:13.429285    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 13:15:14.433292 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:14.433261    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:15:14.434503 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:14.434478    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:15:24.663882 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:24.663839    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj" podUID="bf83370d-4442-442f-82a1-fa013ef3e761" containerName="main" containerID="cri-o://bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21" gracePeriod=45
Sep 15 13:15:24.916521 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:24.916461    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:15:25.015389 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.015360    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-dshm\") pod \"bf83370d-4442-442f-82a1-fa013ef3e761\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") "
Sep 15 13:15:25.015565 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.015452    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-tmp-dir\") pod \"bf83370d-4442-442f-82a1-fa013ef3e761\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") "
Sep 15 13:15:25.015616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.015584    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-model-cache\") pod \"bf83370d-4442-442f-82a1-fa013ef3e761\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") "
Sep 15 13:15:25.015664 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.015615    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtrlg\" (UniqueName: \"kubernetes.io/projected/bf83370d-4442-442f-82a1-fa013ef3e761-kube-api-access-jtrlg\") pod \"bf83370d-4442-442f-82a1-fa013ef3e761\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") "
Sep 15 13:15:25.015664 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.015653    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf83370d-4442-442f-82a1-fa013ef3e761-tls-certs\") pod \"bf83370d-4442-442f-82a1-fa013ef3e761\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") "
Sep 15 13:15:25.015774 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.015682    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-home\") pod \"bf83370d-4442-442f-82a1-fa013ef3e761\" (UID: \"bf83370d-4442-442f-82a1-fa013ef3e761\") "
Sep 15 13:15:25.015774 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.015753    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf83370d-4442-442f-82a1-fa013ef3e761" (UID: "bf83370d-4442-442f-82a1-fa013ef3e761"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:25.015886 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.015864    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf83370d-4442-442f-82a1-fa013ef3e761" (UID: "bf83370d-4442-442f-82a1-fa013ef3e761"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:25.016069 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.016025    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-home" (OuterVolumeSpecName: "home") pod "bf83370d-4442-442f-82a1-fa013ef3e761" (UID: "bf83370d-4442-442f-82a1-fa013ef3e761"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:25.016249 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.016232    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-model-cache\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:25.016335 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.016252    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-tmp-dir\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:25.018073 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.018004    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf83370d-4442-442f-82a1-fa013ef3e761-kube-api-access-jtrlg" (OuterVolumeSpecName: "kube-api-access-jtrlg") pod "bf83370d-4442-442f-82a1-fa013ef3e761" (UID: "bf83370d-4442-442f-82a1-fa013ef3e761"). InnerVolumeSpecName "kube-api-access-jtrlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:15:25.018184 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.018110    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf83370d-4442-442f-82a1-fa013ef3e761-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf83370d-4442-442f-82a1-fa013ef3e761" (UID: "bf83370d-4442-442f-82a1-fa013ef3e761"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:15:25.018184 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.018124    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-dshm" (OuterVolumeSpecName: "dshm") pod "bf83370d-4442-442f-82a1-fa013ef3e761" (UID: "bf83370d-4442-442f-82a1-fa013ef3e761"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:25.117499 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.117470    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtrlg\" (UniqueName: \"kubernetes.io/projected/bf83370d-4442-442f-82a1-fa013ef3e761-kube-api-access-jtrlg\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:25.117499 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.117494    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf83370d-4442-442f-82a1-fa013ef3e761-tls-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:25.117499 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.117503    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-home\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:25.117720 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.117511    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf83370d-4442-442f-82a1-fa013ef3e761-dshm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:25.362994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.362961    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf83370d-4442-442f-82a1-fa013ef3e761" containerID="bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21" exitCode=0
Sep 15 13:15:25.362994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.363000    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"
Sep 15 13:15:25.363274 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.363000    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj" event={"ID":"bf83370d-4442-442f-82a1-fa013ef3e761","Type":"ContainerDied","Data":"bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21"}
Sep 15 13:15:25.363274 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.363147    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj" event={"ID":"bf83370d-4442-442f-82a1-fa013ef3e761","Type":"ContainerDied","Data":"7a20efbe95d1aef15de7938a3e16d5b69477d082c0de254c12a62d384ac8dcc0"}
Sep 15 13:15:25.363274 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.363177    2562 scope.go:117] "RemoveContainer" containerID="bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21"
Sep 15 13:15:25.373442 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.373424    2562 scope.go:117] "RemoveContainer" containerID="bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21"
Sep 15 13:15:25.373727 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:15:25.373704    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21\": container with ID starting with bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21 not found: ID does not exist" containerID="bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21"
Sep 15 13:15:25.373794 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.373738    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21"} err="failed to get container status \"bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21\": rpc error: code = NotFound desc = could not find container \"bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21\": container with ID starting with bde766f96da4d36f2df970ff9915d00068c50058f4ab529846c4d3ccee0e8b21 not found: ID does not exist"
Sep 15 13:15:25.390020 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.389996    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"]
Sep 15 13:15:25.397674 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.397639    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7e508e61/e2e-lifecycle-model-7e508e61-kserve-866c9486ff-dmxkj"]
Sep 15 13:15:25.505286 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:25.505251    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:15:26.756917 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:26.756872    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf83370d-4442-442f-82a1-fa013ef3e761" path="/var/lib/kubelet/pods/bf83370d-4442-442f-82a1-fa013ef3e761/volumes"
Sep 15 13:15:36.437642 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:36.437607    2562 generic.go:356] "Generic (PLEG): container finished" podID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerID="f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f" exitCode=0
Sep 15 13:15:36.438151 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:36.437687    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" event={"ID":"3f6e1bad-60f7-43a4-b48d-aa4965c057c2","Type":"ContainerDied","Data":"f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f"}
Sep 15 13:15:36.438350 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:36.438332    2562 scope.go:117] "RemoveContainer" containerID="f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f"
Sep 15 13:15:36.439735 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:36.439702    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:36.439837 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:36.439768    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:36.439837 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:36.439798    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:15:37.449827 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:37.449778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" event={"ID":"3f6e1bad-60f7-43a4-b48d-aa4965c057c2","Type":"ContainerStarted","Data":"8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558"}
Sep 15 13:15:37.870077 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:37.869965    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:37.871269 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:37.871241    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 15 13:15:37.871393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:37.871301    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 15 13:15:38.870214 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:38.870180    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 15 13:15:38.870699 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:38.870258    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 15 13:15:39.871117 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:39.871077    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 15 13:15:39.871596 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:39.871140    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 15 13:15:40.871058 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:40.870998    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 15 13:15:40.871234 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:40.871077    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 15 13:15:41.276463 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:41.276429    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:41.871249 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:41.871211    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 15 13:15:41.871622 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:41.871292    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 15 13:15:42.874678 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:42.874640    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:42.875604 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:42.875584    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:15:43.073952 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:43.073922    2562 scope.go:117] "RemoveContainer" containerID="dd58b3f95d398f2a7710cdfee1c58917e316e70bae94e7e979dbb4ff3ab83e38"
Sep 15 13:15:49.704121 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.704085    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"]
Sep 15 13:15:49.704819 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.704719    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf83370d-4442-442f-82a1-fa013ef3e761" containerName="main"
Sep 15 13:15:49.704819 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.704735    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf83370d-4442-442f-82a1-fa013ef3e761" containerName="main"
Sep 15 13:15:49.704819 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.704800    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf83370d-4442-442f-82a1-fa013ef3e761" containerName="main"
Sep 15 13:15:49.707713 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.707691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.710870 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.710844    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-aa8ef1\"/\"auth-test-model-aa8ef1-kserve-self-signed-certs\""
Sep 15 13:15:49.712276 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.712255    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-aa8ef1\"/\"kube-root-ca.crt\""
Sep 15 13:15:49.713367 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.713345    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-aa8ef1\"/\"openshift-service-ca.crt\""
Sep 15 13:15:49.713469 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.713394    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-aa8ef1\"/\"default-dockercfg-lcrcn\""
Sep 15 13:15:49.721626 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.721601    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"]
Sep 15 13:15:49.806365 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.806331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-model-cache\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.806365 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.806366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-home\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.806560 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.806482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tmp-dir\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.806560 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.806507    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tls-certs\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.806638 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.806592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-dshm\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.806675 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.806635    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgq8\" (UniqueName: \"kubernetes.io/projected/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-kube-api-access-xwgq8\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.912140 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.912096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-dshm\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.913043 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.913000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgq8\" (UniqueName: \"kubernetes.io/projected/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-kube-api-access-xwgq8\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.913169 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.913144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-model-cache\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.913502 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.913477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-model-cache\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.913595 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.913173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-home\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.913843 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.913821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-home\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.913934 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.913920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tmp-dir\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.913996 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.913944    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tls-certs\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.914696 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.914662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tmp-dir\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.917437 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.917393    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-dshm\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.922905 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.922864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tls-certs\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:49.929153 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:49.929113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgq8\" (UniqueName: \"kubernetes.io/projected/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-kube-api-access-xwgq8\") pod \"auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") " pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:50.031516 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:50.031432    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:15:50.308896 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:50.308864    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"]
Sep 15 13:15:50.309404 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:15:50.309374    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fe70d49_ed1c_4a74_b7b2_9ce75abc77b3.slice/crio-c415b0a30050cc07b9189bbda8d357cd821545b1e04df1dba3b98ef5528c2588 WatchSource:0}: Error finding container c415b0a30050cc07b9189bbda8d357cd821545b1e04df1dba3b98ef5528c2588: Status 404 returned error can't find the container with id c415b0a30050cc07b9189bbda8d357cd821545b1e04df1dba3b98ef5528c2588
Sep 15 13:15:50.533938 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:50.533904    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt" event={"ID":"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3","Type":"ContainerStarted","Data":"4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80"}
Sep 15 13:15:50.533938 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:50.533950    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt" event={"ID":"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3","Type":"ContainerStarted","Data":"c415b0a30050cc07b9189bbda8d357cd821545b1e04df1dba3b98ef5528c2588"}
Sep 15 13:15:50.558355 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:50.558252    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt" podStartSLOduration=1.5582367449999999 podStartE2EDuration="1.558236745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:15:50.556723808 +0000 UTC m=+1868.349425541" watchObservedRunningTime="2026-09-15 13:15:50.558236745 +0000 UTC m=+1868.350938475"
Sep 15 13:15:53.080766 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:53.080728    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"]
Sep 15 13:15:53.081195 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:53.081010    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" containerID="cri-o://318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b" gracePeriod=30
Sep 15 13:15:56.323110 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:56.323066    2562 patch_prober.go:28] interesting pod/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 13:15:56.323502 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:56.323172    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 13:15:57.882920 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:57.882891    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:15:58.363531 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.363508    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:15:58.514147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514026    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-envoy\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514104    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4jll\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-kube-api-access-d4jll\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514151    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-socket\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514215    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58bbef12-2135-43f6-90e3-71cf9f6a538f-istiod-ca-cert\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514238    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-podinfo\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514265    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-data\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514304    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-certs\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514390 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514340    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-credential-socket\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514503    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:58.514700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514565    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58bbef12-2135-43f6-90e3-71cf9f6a538f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:15:58.514700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514588    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-data" (OuterVolumeSpecName: "istio-data") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:58.514700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514606    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:58.514700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514660    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-token\") pod \"58bbef12-2135-43f6-90e3-71cf9f6a538f\" (UID: \"58bbef12-2135-43f6-90e3-71cf9f6a538f\") "
Sep 15 13:15:58.514947 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.514834    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:58.515236 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.515216    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.515330 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.515238    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58bbef12-2135-43f6-90e3-71cf9f6a538f-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.515330 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.515251    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.515330 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.515262    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.515330 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.515274    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.516385 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.516363    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-kube-api-access-d4jll" (OuterVolumeSpecName: "kube-api-access-d4jll") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "kube-api-access-d4jll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:15:58.516385 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.516372    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:15:58.516651 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.516634    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:15:58.516795 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.516781    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-token" (OuterVolumeSpecName: "istio-token") pod "58bbef12-2135-43f6-90e3-71cf9f6a538f" (UID: "58bbef12-2135-43f6-90e3-71cf9f6a538f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:15:58.585738 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.585708    2562 generic.go:356] "Generic (PLEG): container finished" podID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerID="318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b" exitCode=0
Sep 15 13:15:58.585898 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.585778    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"
Sep 15 13:15:58.585898 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.585786    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" event={"ID":"58bbef12-2135-43f6-90e3-71cf9f6a538f","Type":"ContainerDied","Data":"318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b"}
Sep 15 13:15:58.585898 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.585865    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln" event={"ID":"58bbef12-2135-43f6-90e3-71cf9f6a538f","Type":"ContainerDied","Data":"2abd4d7ce630fae407cc5b4233a6c7896f51858cd60308df386e4d68e1f773f5"}
Sep 15 13:15:58.585898 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.585889    2562 scope.go:117] "RemoveContainer" containerID="318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b"
Sep 15 13:15:58.595610 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.595593    2562 scope.go:117] "RemoveContainer" containerID="e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe"
Sep 15 13:15:58.603961 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.603943    2562 scope.go:117] "RemoveContainer" containerID="318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b"
Sep 15 13:15:58.604236 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:15:58.604218    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b\": container with ID starting with 318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b not found: ID does not exist" containerID="318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b"
Sep 15 13:15:58.604306 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.604244    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b"} err="failed to get container status \"318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b\": rpc error: code = NotFound desc = could not find container \"318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b\": container with ID starting with 318563c3d0ce4bd5c449123e74c69703e694ee8fa3fc2593a6803cbc7747e49b not found: ID does not exist"
Sep 15 13:15:58.604306 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.604260    2562 scope.go:117] "RemoveContainer" containerID="e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe"
Sep 15 13:15:58.604516 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:15:58.604499    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe\": container with ID starting with e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe not found: ID does not exist" containerID="e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe"
Sep 15 13:15:58.604560 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.604523    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe"} err="failed to get container status \"e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe\": rpc error: code = NotFound desc = could not find container \"e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe\": container with ID starting with e97a126c4bd165bdf9020a6eebd152d40bd829984b057d660ae936b90060eabe not found: ID does not exist"
Sep 15 13:15:58.610198 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.610173    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"]
Sep 15 13:15:58.612686 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.612662    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7e508e61-openshift-default-8545d5c8d-p8cln"]
Sep 15 13:15:58.616420 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.616400    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.616420 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.616422    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.616542 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.616431    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4jll\" (UniqueName: \"kubernetes.io/projected/58bbef12-2135-43f6-90e3-71cf9f6a538f-kube-api-access-d4jll\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.616542 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.616440    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58bbef12-2135-43f6-90e3-71cf9f6a538f-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:15:58.757573 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:15:58.757538    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" path="/var/lib/kubelet/pods/58bbef12-2135-43f6-90e3-71cf9f6a538f/volumes"
Sep 15 13:16:00.032641 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:16:00.032599    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:16:00.033141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:16:00.032661    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:16:00.045496 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:16:00.045469    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:16:00.611348 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:16:00.611318    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:16:26.101861 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:16:26.101829    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:16:43.107628 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:16:43.107594    2562 scope.go:117] "RemoveContainer" containerID="ef25b984137edbc0ff154159447d7e141c816f76fe57757be9370b60372f49de"
Sep 15 13:17:00.825708 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:00.825668    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"]
Sep 15 13:17:00.826700 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:00.826641    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" containerID="cri-o://94a30823d271f6fa1f870d462e86f645b4da338c757774323a8276869d62f195" gracePeriod=30
Sep 15 13:17:03.344533 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:03.344491    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 13:17:03.344916 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:03.344550    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 13:17:06.009710 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.009680    2562 generic.go:356] "Generic (PLEG): container finished" podID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerID="94a30823d271f6fa1f870d462e86f645b4da338c757774323a8276869d62f195" exitCode=0
Sep 15 13:17:06.010134 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.009735    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" event={"ID":"c527b9ee-4b9b-489e-b168-d985cbf47219","Type":"ContainerDied","Data":"94a30823d271f6fa1f870d462e86f645b4da338c757774323a8276869d62f195"}
Sep 15 13:17:06.010134 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.009798    2562 scope.go:117] "RemoveContainer" containerID="0cc3e9026233ee8f798df7f436c675c8f66500e08892a0f43a871b65f67968c5"
Sep 15 13:17:06.091297 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.091239    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 13:17:06.188524 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.188487    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:17:06.244227 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244194    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-podinfo\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244380 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244235    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-socket\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244380 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244276    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-data\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244508 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c527b9ee-4b9b-489e-b168-d985cbf47219-istiod-ca-cert\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244508 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244440    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-envoy\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244619 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244524    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58dhb\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-kube-api-access-58dhb\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244619 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244534    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-data" (OuterVolumeSpecName: "istio-data") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:17:06.244619 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244551    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:17:06.244619 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244560    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-credential-socket\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244834 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244641    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-token\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244834 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-certs\") pod \"c527b9ee-4b9b-489e-b168-d985cbf47219\" (UID: \"c527b9ee-4b9b-489e-b168-d985cbf47219\") "
Sep 15 13:17:06.244834 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244818    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:17:06.244971 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.244842    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c527b9ee-4b9b-489e-b168-d985cbf47219-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:17:06.245114 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.245087    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:17:06.245217 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.245202    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:06.245281 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.245218    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:06.245281 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.245227    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c527b9ee-4b9b-489e-b168-d985cbf47219-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:06.245281 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.245236    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:06.245281 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.245249    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:06.246473 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.246447    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:17:06.246723 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.246698    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-token" (OuterVolumeSpecName: "istio-token") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:17:06.247024 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.247007    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:17:06.247113 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.247020    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-kube-api-access-58dhb" (OuterVolumeSpecName: "kube-api-access-58dhb") pod "c527b9ee-4b9b-489e-b168-d985cbf47219" (UID: "c527b9ee-4b9b-489e-b168-d985cbf47219"). InnerVolumeSpecName "kube-api-access-58dhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:17:06.346477 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.346396    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58dhb\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-kube-api-access-58dhb\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:06.346477 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.346429    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:06.346477 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.346442    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:06.346477 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:06.346450    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c527b9ee-4b9b-489e-b168-d985cbf47219-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:17:07.017168 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:07.017078    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"
Sep 15 13:17:07.017168 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:07.017086    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk" event={"ID":"c527b9ee-4b9b-489e-b168-d985cbf47219","Type":"ContainerDied","Data":"de81556079892e620cfbfe0d67390afaf07084d62c749d4ecbf0b493c2098e56"}
Sep 15 13:17:07.017168 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:07.017148    2562 scope.go:117] "RemoveContainer" containerID="94a30823d271f6fa1f870d462e86f645b4da338c757774323a8276869d62f195"
Sep 15 13:17:07.036374 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:07.036348    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"]
Sep 15 13:17:07.040372 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:07.040338    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-6c9f0b-openshift-default-74cff664f6-rd6wk"]
Sep 15 13:17:08.757185 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:17:08.757150    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" path="/var/lib/kubelet/pods/c527b9ee-4b9b-489e-b168-d985cbf47219/volumes"
Sep 15 13:18:27.147260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:27.147218    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"]
Sep 15 13:18:27.147789 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:27.147429    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" containerID="cri-o://288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038" gracePeriod=30
Sep 15 13:18:31.315927 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:31.315888    2562 patch_prober.go:28] interesting pod/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 13:18:31.316329 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:31.315952    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 13:18:32.392683 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.392657    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 13:18:32.496605 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.496572    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:18:32.502326 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.502296    2562 generic.go:356] "Generic (PLEG): container finished" podID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerID="288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038" exitCode=0
Sep 15 13:18:32.502478 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.502341    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"
Sep 15 13:18:32.502478 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.502383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" event={"ID":"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8","Type":"ContainerDied","Data":"288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038"}
Sep 15 13:18:32.502478 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.502440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p" event={"ID":"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8","Type":"ContainerDied","Data":"ee7db6ece36a79f1b91196d010ba41a278690d515b3691864180fc701ccf8031"}
Sep 15 13:18:32.502669 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.502472    2562 scope.go:117] "RemoveContainer" containerID="288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038"
Sep 15 13:18:32.511901 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.511878    2562 scope.go:117] "RemoveContainer" containerID="cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b"
Sep 15 13:18:32.520095 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.520077    2562 scope.go:117] "RemoveContainer" containerID="288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038"
Sep 15 13:18:32.520409 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:18:32.520388    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038\": container with ID starting with 288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038 not found: ID does not exist" containerID="288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038"
Sep 15 13:18:32.520500 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.520413    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038"} err="failed to get container status \"288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038\": rpc error: code = NotFound desc = could not find container \"288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038\": container with ID starting with 288883073ef38b9f5f36527217fadf3fda55247db33e861261af511adb0a8038 not found: ID does not exist"
Sep 15 13:18:32.520500 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.520428    2562 scope.go:117] "RemoveContainer" containerID="cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b"
Sep 15 13:18:32.520699 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:18:32.520681    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b\": container with ID starting with cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b not found: ID does not exist" containerID="cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b"
Sep 15 13:18:32.520757 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.520702    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b"} err="failed to get container status \"cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b\": rpc error: code = NotFound desc = could not find container \"cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b\": container with ID starting with cb17150f0d00c5551cf389e86cd3fc2f797c820460fc14fc00baf323c0f93c2b not found: ID does not exist"
Sep 15 13:18:32.553180 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553153    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istiod-ca-cert\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553299 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553263    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-certs\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553341 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553299    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-socket\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553362    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-data\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553446 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clgj4\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-kube-api-access-clgj4\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553446 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553418    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-podinfo\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553549 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553447    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-token\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553549 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553479    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-credential-socket\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553549 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553506    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-envoy\") pod \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\" (UID: \"8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8\") "
Sep 15 13:18:32.553697 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:18:32.553697 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553617    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:18:32.553697 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553630    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:18:32.553881 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553854    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:18:32.553969 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553909    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:18:32.554041 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553966    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.554041 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.553988    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.554041 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.554003    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.554041 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.554016    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.555565 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.555541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:18:32.555565 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.555558    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:18:32.555720 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.555655    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-kube-api-access-clgj4" (OuterVolumeSpecName: "kube-api-access-clgj4") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "kube-api-access-clgj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:18:32.555720 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.555685    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" (UID: "8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:18:32.655430 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.655398    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.655430 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.655430    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clgj4\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-kube-api-access-clgj4\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.655658 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.655445    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.655658 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.655458    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.655658 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.655469    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:18:32.819346 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.819313    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"]
Sep 15 13:18:32.823370 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:32.823336    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-2af9a3-openshift-default-549746dcfb-l4d9p"]
Sep 15 13:18:34.757166 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:18:34.757132    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" path="/var/lib/kubelet/pods/8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8/volumes"
Sep 15 13:19:42.900803 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:19:42.900778    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:19:42.904524 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:19:42.904495    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:19:42.907019 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:19:42.906999    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:19:42.913597 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:19:42.913576    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:21:47.003312 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003280    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"]
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003735    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003747    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003765    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003771    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003777    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003786    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003796    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003802    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003811    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003815    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003889    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003899    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec96a3c-240c-476b-ac5c-a3d0fe8b1ef8" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003905    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003911    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c527b9ee-4b9b-489e-b168-d985cbf47219" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003921    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy"
Sep 15 13:21:47.005840 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.003926    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy"
Sep 15 13:21:47.007089 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.007074    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.009862 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.009839    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-650c57-openshift-default-dockercfg-92nvn\""
Sep 15 13:21:47.029513 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.029484    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"]
Sep 15 13:21:47.173985 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.173950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvx6r\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-kube-api-access-zvx6r\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.174168 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.174130    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-data\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.174213 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.174171    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-credential-socket\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.174213 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.174202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-socket\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.174278 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.174217    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-envoy\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.174278 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.174249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cdf1993-f825-4fac-8445-e436b818319e-istio-podinfo\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.174348 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.174287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-istio-token\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.174348 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.174324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cdf1993-f825-4fac-8445-e436b818319e-istiod-ca-cert\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.174412 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.174364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-certs\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.275880 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.275800    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-data\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.275880 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.275837    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-credential-socket\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.275880 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.275862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-socket\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276193 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.275881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-envoy\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276193 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.275906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cdf1993-f825-4fac-8445-e436b818319e-istio-podinfo\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276193 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.275923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-istio-token\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276193 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.275948    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cdf1993-f825-4fac-8445-e436b818319e-istiod-ca-cert\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276409 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.276186    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-certs\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276409 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.276271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-socket\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276409 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.276319    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-credential-socket\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276409 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.276332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvx6r\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-kube-api-access-zvx6r\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276631 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.276594    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-data\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276693 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.276646    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-certs\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.276738 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.276677    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cdf1993-f825-4fac-8445-e436b818319e-istiod-ca-cert\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.278377 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.278354    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-envoy\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.278519 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.278503    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cdf1993-f825-4fac-8445-e436b818319e-istio-podinfo\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.284229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.284202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvx6r\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-kube-api-access-zvx6r\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.284229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.284221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-istio-token\") pod \"e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") " pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.317773 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.317749    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:47.598870 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.598836    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"]
Sep 15 13:21:47.599823 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:21:47.599792    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cdf1993_f825_4fac_8445_e436b818319e.slice/crio-2e129982421fb765af92d12f9e4d1cfc549d459816795ef051029ecee95a21e3 WatchSource:0}: Error finding container 2e129982421fb765af92d12f9e4d1cfc549d459816795ef051029ecee95a21e3: Status 404 returned error can't find the container with id 2e129982421fb765af92d12f9e4d1cfc549d459816795ef051029ecee95a21e3
Sep 15 13:21:47.602295 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.602257    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:21:47.602366 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.602350    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:21:47.602412 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:47.602397    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:21:48.598867 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:48.598832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" event={"ID":"0cdf1993-f825-4fac-8445-e436b818319e","Type":"ContainerStarted","Data":"7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7"}
Sep 15 13:21:48.599270 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:48.598882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" event={"ID":"0cdf1993-f825-4fac-8445-e436b818319e","Type":"ContainerStarted","Data":"2e129982421fb765af92d12f9e4d1cfc549d459816795ef051029ecee95a21e3"}
Sep 15 13:21:48.619824 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:48.619778    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" podStartSLOduration=2.619762385 podStartE2EDuration="2.619762385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:21:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:21:48.618769573 +0000 UTC m=+2226.411471327" watchObservedRunningTime="2026-09-15 13:21:48.619762385 +0000 UTC m=+2226.412464110"
Sep 15 13:21:49.318131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:49.318098    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:49.323571 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:49.323542    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:49.605131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:49.605050    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:49.606011 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:49.605990    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:21:53.580593 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.580556    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"]
Sep 15 13:21:53.581300 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.581267    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy"
Sep 15 13:21:53.581300 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.581294    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="58bbef12-2135-43f6-90e3-71cf9f6a538f" containerName="istio-proxy"
Sep 15 13:21:53.584736 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.584715    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.587505 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.587481    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-650c57-serving-cert\""
Sep 15 13:21:53.597187 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.597165    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"]
Sep 15 13:21:53.635928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.635897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26bef37e-4751-43b8-ad13-4d18222ce8ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.636096 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.635970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9jrv\" (UniqueName: \"kubernetes.io/projected/26bef37e-4751-43b8-ad13-4d18222ce8ba-kube-api-access-v9jrv\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.636096 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.636004    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-metrics-certs\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.636096 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.636026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-maas-api-tls\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.737122 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.737093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26bef37e-4751-43b8-ad13-4d18222ce8ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.737322 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.737170    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9jrv\" (UniqueName: \"kubernetes.io/projected/26bef37e-4751-43b8-ad13-4d18222ce8ba-kube-api-access-v9jrv\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.737322 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.737210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-metrics-certs\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.737322 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.737235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-maas-api-tls\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.737823 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.737799    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26bef37e-4751-43b8-ad13-4d18222ce8ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.739690 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.739667    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-maas-api-tls\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.739804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.739708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-metrics-certs\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.745179 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.745159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9jrv\" (UniqueName: \"kubernetes.io/projected/26bef37e-4751-43b8-ad13-4d18222ce8ba-kube-api-access-v9jrv\") pod \"maas-api-e2e-api-a-650c57-d657c8998-fgfpq\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:53.898519 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:53.898441    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:54.177833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:54.177799    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"]
Sep 15 13:21:54.179256 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:21:54.179214    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26bef37e_4751_43b8_ad13_4d18222ce8ba.slice/crio-b6fdf1fd8e2e19fe140a87494d8f2eb58e07e5a1b2b699c3a03b31a2a516b4ce WatchSource:0}: Error finding container b6fdf1fd8e2e19fe140a87494d8f2eb58e07e5a1b2b699c3a03b31a2a516b4ce: Status 404 returned error can't find the container with id b6fdf1fd8e2e19fe140a87494d8f2eb58e07e5a1b2b699c3a03b31a2a516b4ce
Sep 15 13:21:54.180850 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:54.180831    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 13:21:54.639075 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:54.639002    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq" event={"ID":"26bef37e-4751-43b8-ad13-4d18222ce8ba","Type":"ContainerStarted","Data":"2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0"}
Sep 15 13:21:54.639407 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:54.639078    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:21:54.639407 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:54.639103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq" event={"ID":"26bef37e-4751-43b8-ad13-4d18222ce8ba","Type":"ContainerStarted","Data":"b6fdf1fd8e2e19fe140a87494d8f2eb58e07e5a1b2b699c3a03b31a2a516b4ce"}
Sep 15 13:21:54.660453 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:21:54.660413    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq" podStartSLOduration=1.266292933 podStartE2EDuration="1.660397275s" totalImagesPullingTime="394.104342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:21:54.657721595 +0000 UTC m=+2232.450423324" watchObservedRunningTime="2026-09-15 13:21:54.660397275 +0000 UTC m=+2232.453099039"
Sep 15 13:22:00.651053 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:00.650995    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:22:03.397047 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:03.397006    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:22:05.967457 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:05.967379    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"]
Sep 15 13:22:05.972093 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:05.972069    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:05.974901 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:05.974880    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-92f861-openshift-default-dockercfg-n9zwc\""
Sep 15 13:22:05.985638 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:05.985611    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"]
Sep 15 13:22:06.069537 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-socket\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.069718 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0632853-bc89-47ed-9622-9585abd8b5da-istio-podinfo\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.069718 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0632853-bc89-47ed-9622-9585abd8b5da-istiod-ca-cert\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.069718 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sph59\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-kube-api-access-sph59\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.069897 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-istio-token\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.069897 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-data\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.069897 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-credential-socket\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.069897 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-envoy\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.069897 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.069892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-certs\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.170845 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.170803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sph59\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-kube-api-access-sph59\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.170854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-istio-token\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.170883    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-data\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.170907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-credential-socket\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.170930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-envoy\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.170957    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-certs\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.171019    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-socket\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171367 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.171107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0632853-bc89-47ed-9622-9585abd8b5da-istio-podinfo\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171367 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.171137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0632853-bc89-47ed-9622-9585abd8b5da-istiod-ca-cert\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171531 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.171451    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-data\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171531 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.171464    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-credential-socket\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171642 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.171614    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-certs\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.171888 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.171865    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0632853-bc89-47ed-9622-9585abd8b5da-istiod-ca-cert\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.172123 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.172074    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-socket\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.175986 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.174987    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-envoy\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.175986 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.175141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0632853-bc89-47ed-9622-9585abd8b5da-istio-podinfo\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.179894 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.179866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-istio-token\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.180471 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.180455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sph59\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-kube-api-access-sph59\") pod \"e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") " pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.287062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.286938    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:06.605279 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.605249    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"]
Sep 15 13:22:06.606386 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:22:06.606361    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0632853_bc89_47ed_9622_9585abd8b5da.slice/crio-ad5d72d45821bffb3996621558dea5572116e58545bce04103bfc01647df48be WatchSource:0}: Error finding container ad5d72d45821bffb3996621558dea5572116e58545bce04103bfc01647df48be: Status 404 returned error can't find the container with id ad5d72d45821bffb3996621558dea5572116e58545bce04103bfc01647df48be
Sep 15 13:22:06.608911 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.608872    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:06.608999 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.608958    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:06.609067 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.609007    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:06.725564 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.724703    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" event={"ID":"f0632853-bc89-47ed-9622-9585abd8b5da","Type":"ContainerStarted","Data":"59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac"}
Sep 15 13:22:06.725564 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.724768    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" event={"ID":"f0632853-bc89-47ed-9622-9585abd8b5da","Type":"ContainerStarted","Data":"ad5d72d45821bffb3996621558dea5572116e58545bce04103bfc01647df48be"}
Sep 15 13:22:06.755224 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:06.752712    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" podStartSLOduration=1.752692623 podStartE2EDuration="1.752692623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:22:06.751698627 +0000 UTC m=+2244.544400359" watchObservedRunningTime="2026-09-15 13:22:06.752692623 +0000 UTC m=+2244.545394354"
Sep 15 13:22:07.287538 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:07.287505    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:07.292152 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:07.292123    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:07.734596 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:07.734560    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:07.739421 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:07.739386    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:10.968949 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:10.968917    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"]
Sep 15 13:22:10.973247 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:10.973192    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:10.975910 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:10.975890    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-92f861-serving-cert\""
Sep 15 13:22:10.988164 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:10.988142    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"]
Sep 15 13:22:11.025519 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.025482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-metrics-certs\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.025795 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.025778    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-maas-api-tls\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.026013 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.025996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvr5l\" (UniqueName: \"kubernetes.io/projected/3c7f46b3-93ce-4355-913f-d974f590ba96-kube-api-access-nvr5l\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.026188 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.026172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7f46b3-93ce-4355-913f-d974f590ba96-openshift-service-ca\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.127931 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.127877    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-metrics-certs\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.128141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.127964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-maas-api-tls\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.128141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.128081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvr5l\" (UniqueName: \"kubernetes.io/projected/3c7f46b3-93ce-4355-913f-d974f590ba96-kube-api-access-nvr5l\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.128141 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.128137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7f46b3-93ce-4355-913f-d974f590ba96-openshift-service-ca\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.128987 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.128956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7f46b3-93ce-4355-913f-d974f590ba96-openshift-service-ca\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.130838 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.130806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-metrics-certs\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.130965 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.130864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-maas-api-tls\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.136912 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.136886    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvr5l\" (UniqueName: \"kubernetes.io/projected/3c7f46b3-93ce-4355-913f-d974f590ba96-kube-api-access-nvr5l\") pod \"maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.286089 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.285993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:11.575410 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.575378    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"]
Sep 15 13:22:11.575675 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:22:11.575643    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7f46b3_93ce_4355_913f_d974f590ba96.slice/crio-b51498624fedb331e4312bd4b9112c9a6b7c27e6c8ba5894c55574e0d56fb6a6 WatchSource:0}: Error finding container b51498624fedb331e4312bd4b9112c9a6b7c27e6c8ba5894c55574e0d56fb6a6: Status 404 returned error can't find the container with id b51498624fedb331e4312bd4b9112c9a6b7c27e6c8ba5894c55574e0d56fb6a6
Sep 15 13:22:11.768196 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:11.768155    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd" event={"ID":"3c7f46b3-93ce-4355-913f-d974f590ba96","Type":"ContainerStarted","Data":"b51498624fedb331e4312bd4b9112c9a6b7c27e6c8ba5894c55574e0d56fb6a6"}
Sep 15 13:22:12.775065 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:12.775008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd" event={"ID":"3c7f46b3-93ce-4355-913f-d974f590ba96","Type":"ContainerStarted","Data":"bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a"}
Sep 15 13:22:12.775441 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:12.775194    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:12.796654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:12.796608    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd" podStartSLOduration=2.3025916779999998 podStartE2EDuration="2.796591679s" totalImagesPullingTime="494.000001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:22:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:22:12.793211817 +0000 UTC m=+2250.585913549" watchObservedRunningTime="2026-09-15 13:22:12.796591679 +0000 UTC m=+2250.589293410"
Sep 15 13:22:14.789634 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:14.789561    2562 generic.go:356] "Generic (PLEG): container finished" podID="0cdf1993-f825-4fac-8445-e436b818319e" containerID="7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7" exitCode=0
Sep 15 13:22:14.790044 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:14.789634    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" event={"ID":"0cdf1993-f825-4fac-8445-e436b818319e","Type":"ContainerDied","Data":"7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7"}
Sep 15 13:22:14.790126 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:14.790111    2562 scope.go:117] "RemoveContainer" containerID="7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7"
Sep 15 13:22:14.791348 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:14.791298    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:14.791423 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:14.791363    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:14.791423 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:14.791393    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:15.797109 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:15.797069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" event={"ID":"0cdf1993-f825-4fac-8445-e436b818319e","Type":"ContainerStarted","Data":"7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24"}
Sep 15 13:22:16.318265 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:16.318224    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:22:16.319358 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:16.319332    2562 patch_prober.go:28] interesting pod/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 13:22:16.319478 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:16.319381    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 13:22:17.318454 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:17.318420    2562 patch_prober.go:28] interesting pod/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 13:22:17.318889 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:17.318472    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 13:22:18.318207 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:18.318171    2562 patch_prober.go:28] interesting pod/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 13:22:18.318508 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:18.318243    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 13:22:18.788226 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:18.788197    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:22:19.318208 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:19.318171    2562 patch_prober.go:28] interesting pod/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 13:22:19.318400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:19.318240    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 13:22:19.605286 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:19.605196    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:22:20.318377 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:20.318339    2562 patch_prober.go:28] interesting pod/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 13:22:20.318550 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:20.318392    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 13:22:21.322701 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:21.322669    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:22:21.323918 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:21.323894    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:22:26.391843 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:26.391808    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:22:36.933684 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:36.933648    2562 generic.go:356] "Generic (PLEG): container finished" podID="f0632853-bc89-47ed-9622-9585abd8b5da" containerID="59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac" exitCode=0
Sep 15 13:22:36.934101 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:36.933726    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" event={"ID":"f0632853-bc89-47ed-9622-9585abd8b5da","Type":"ContainerDied","Data":"59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac"}
Sep 15 13:22:36.934213 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:36.934196    2562 scope.go:117] "RemoveContainer" containerID="59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac"
Sep 15 13:22:36.935493 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:36.935465    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:36.935578 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:36.935534    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:36.935578 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:36.935566    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 13:22:37.735091 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:37.735050    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:37.946927 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:37.946880    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" event={"ID":"f0632853-bc89-47ed-9622-9585abd8b5da","Type":"ContainerStarted","Data":"9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693"}
Sep 15 13:22:38.288089 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:38.288024    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:38.289285 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:38.289262    2562 patch_prober.go:28] interesting pod/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 15 13:22:38.289360 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:38.289314    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 15 13:22:39.288457 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:39.288423    2562 patch_prober.go:28] interesting pod/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 15 13:22:39.288865 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:39.288482    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 15 13:22:40.288362 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:40.288324    2562 patch_prober.go:28] interesting pod/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 15 13:22:40.288536 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:40.288384    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 15 13:22:41.288456 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:41.288418    2562 patch_prober.go:28] interesting pod/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 15 13:22:41.288640 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:41.288484    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 15 13:22:42.288458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:42.288417    2562 patch_prober.go:28] interesting pod/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 15 13:22:42.288629 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:42.288501    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 15 13:22:43.292379 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:43.292346    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:43.292859 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:43.292750    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:43.293417 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:43.293400    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:22:52.345855 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.345814    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"]
Sep 15 13:22:52.346680 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.346641    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq" podUID="26bef37e-4751-43b8-ad13-4d18222ce8ba" containerName="maas-api" containerID="cri-o://2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0" gracePeriod=30
Sep 15 13:22:52.604126 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.604060    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:22:52.704867 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.704837    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9jrv\" (UniqueName: \"kubernetes.io/projected/26bef37e-4751-43b8-ad13-4d18222ce8ba-kube-api-access-v9jrv\") pod \"26bef37e-4751-43b8-ad13-4d18222ce8ba\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") "
Sep 15 13:22:52.704867 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.704872    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26bef37e-4751-43b8-ad13-4d18222ce8ba-openshift-service-ca\") pod \"26bef37e-4751-43b8-ad13-4d18222ce8ba\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") "
Sep 15 13:22:52.705170 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.704917    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-metrics-certs\") pod \"26bef37e-4751-43b8-ad13-4d18222ce8ba\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") "
Sep 15 13:22:52.705170 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.704964    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-maas-api-tls\") pod \"26bef37e-4751-43b8-ad13-4d18222ce8ba\" (UID: \"26bef37e-4751-43b8-ad13-4d18222ce8ba\") "
Sep 15 13:22:52.705365 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.705323    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26bef37e-4751-43b8-ad13-4d18222ce8ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26bef37e-4751-43b8-ad13-4d18222ce8ba" (UID: "26bef37e-4751-43b8-ad13-4d18222ce8ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:22:52.707185 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.707154    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26bef37e-4751-43b8-ad13-4d18222ce8ba" (UID: "26bef37e-4751-43b8-ad13-4d18222ce8ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:22:52.707299 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.707209    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "26bef37e-4751-43b8-ad13-4d18222ce8ba" (UID: "26bef37e-4751-43b8-ad13-4d18222ce8ba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:22:52.707431 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.707415    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26bef37e-4751-43b8-ad13-4d18222ce8ba-kube-api-access-v9jrv" (OuterVolumeSpecName: "kube-api-access-v9jrv") pod "26bef37e-4751-43b8-ad13-4d18222ce8ba" (UID: "26bef37e-4751-43b8-ad13-4d18222ce8ba"). InnerVolumeSpecName "kube-api-access-v9jrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:22:52.806899 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.806861    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:22:52.807092 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.806911    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9jrv\" (UniqueName: \"kubernetes.io/projected/26bef37e-4751-43b8-ad13-4d18222ce8ba-kube-api-access-v9jrv\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:22:52.807092 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.806929    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26bef37e-4751-43b8-ad13-4d18222ce8ba-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:22:52.807092 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:52.806945    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26bef37e-4751-43b8-ad13-4d18222ce8ba-metrics-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:22:53.039849 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.039814    2562 generic.go:356] "Generic (PLEG): container finished" podID="26bef37e-4751-43b8-ad13-4d18222ce8ba" containerID="2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0" exitCode=0
Sep 15 13:22:53.040013 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.039858    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"
Sep 15 13:22:53.040013 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.039898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq" event={"ID":"26bef37e-4751-43b8-ad13-4d18222ce8ba","Type":"ContainerDied","Data":"2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0"}
Sep 15 13:22:53.040013 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.039948    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq" event={"ID":"26bef37e-4751-43b8-ad13-4d18222ce8ba","Type":"ContainerDied","Data":"b6fdf1fd8e2e19fe140a87494d8f2eb58e07e5a1b2b699c3a03b31a2a516b4ce"}
Sep 15 13:22:53.040013 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.039970    2562 scope.go:117] "RemoveContainer" containerID="2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0"
Sep 15 13:22:53.049209 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.049192    2562 scope.go:117] "RemoveContainer" containerID="2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0"
Sep 15 13:22:53.049512 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:22:53.049491    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0\": container with ID starting with 2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0 not found: ID does not exist" containerID="2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0"
Sep 15 13:22:53.049566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.049520    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0"} err="failed to get container status \"2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0\": rpc error: code = NotFound desc = could not find container \"2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0\": container with ID starting with 2e336ee030fe2f267ddc892eab4ccc2746be4aea83fdc819689b9d468d8cc4c0 not found: ID does not exist"
Sep 15 13:22:53.056858 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.056838    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"]
Sep 15 13:22:53.062358 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:53.062332    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-650c57-d657c8998-fgfpq"]
Sep 15 13:22:54.758021 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:54.757990    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26bef37e-4751-43b8-ad13-4d18222ce8ba" path="/var/lib/kubelet/pods/26bef37e-4751-43b8-ad13-4d18222ce8ba/volumes"
Sep 15 13:22:58.181643 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:58.181609    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:22:59.174423 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:59.174392    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"]
Sep 15 13:22:59.174629 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:22:59.174606    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy" containerID="cri-o://7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24" gracePeriod=30
Sep 15 13:23:04.439266 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.439195    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:23:04.538723 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.538689    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-istio-token\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.538898 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.538833    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-credential-socket\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.538898 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.538880    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-certs\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.539020 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.538927    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-socket\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.539020 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.538952    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cdf1993-f825-4fac-8445-e436b818319e-istio-podinfo\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.539020 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539001    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvx6r\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-kube-api-access-zvx6r\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.539212 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539076    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-data\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.539212 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539100    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-envoy\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.539212 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539130    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cdf1993-f825-4fac-8445-e436b818319e-istiod-ca-cert\") pod \"0cdf1993-f825-4fac-8445-e436b818319e\" (UID: \"0cdf1993-f825-4fac-8445-e436b818319e\") "
Sep 15 13:23:04.539374 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539213    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:04.539374 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539269    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:04.539374 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539285    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:04.539534 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539447    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-data" (OuterVolumeSpecName: "istio-data") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:04.539637 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539614    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cdf1993-f825-4fac-8445-e436b818319e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:23:04.539735 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539716    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:04.539795 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539740    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:04.539795 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539753    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cdf1993-f825-4fac-8445-e436b818319e-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:04.539795 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539767    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:04.539795 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.539791    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:04.541118 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.541088    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cdf1993-f825-4fac-8445-e436b818319e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:23:04.541205 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.541160    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-istio-token" (OuterVolumeSpecName: "istio-token") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:23:04.541580 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.541562    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:04.541632 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.541579    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-kube-api-access-zvx6r" (OuterVolumeSpecName: "kube-api-access-zvx6r") pod "0cdf1993-f825-4fac-8445-e436b818319e" (UID: "0cdf1993-f825-4fac-8445-e436b818319e"). InnerVolumeSpecName "kube-api-access-zvx6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:23:04.641179 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.641144    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cdf1993-f825-4fac-8445-e436b818319e-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:04.641179 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.641173    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:04.641179 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.641181    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cdf1993-f825-4fac-8445-e436b818319e-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:04.641415 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:04.641189    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvx6r\" (UniqueName: \"kubernetes.io/projected/0cdf1993-f825-4fac-8445-e436b818319e-kube-api-access-zvx6r\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:05.111902 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.111818    2562 generic.go:356] "Generic (PLEG): container finished" podID="0cdf1993-f825-4fac-8445-e436b818319e" containerID="7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24" exitCode=0
Sep 15 13:23:05.111902 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.111863    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"
Sep 15 13:23:05.111902 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.111869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" event={"ID":"0cdf1993-f825-4fac-8445-e436b818319e","Type":"ContainerDied","Data":"7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24"}
Sep 15 13:23:05.112217 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.111922    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb" event={"ID":"0cdf1993-f825-4fac-8445-e436b818319e","Type":"ContainerDied","Data":"2e129982421fb765af92d12f9e4d1cfc549d459816795ef051029ecee95a21e3"}
Sep 15 13:23:05.112217 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.111945    2562 scope.go:117] "RemoveContainer" containerID="7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24"
Sep 15 13:23:05.122357 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.122335    2562 scope.go:117] "RemoveContainer" containerID="7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7"
Sep 15 13:23:05.139497 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.139468    2562 scope.go:117] "RemoveContainer" containerID="7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24"
Sep 15 13:23:05.139955 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:23:05.139926    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24\": container with ID starting with 7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24 not found: ID does not exist" containerID="7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24"
Sep 15 13:23:05.140091 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.139974    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24"} err="failed to get container status \"7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24\": rpc error: code = NotFound desc = could not find container \"7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24\": container with ID starting with 7baf3e7c4fc2955adfb451f8ecaa6a8eb5e959d86e98bd7d4d11792282912e24 not found: ID does not exist"
Sep 15 13:23:05.140091 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.140001    2562 scope.go:117] "RemoveContainer" containerID="7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7"
Sep 15 13:23:05.140453 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:23:05.140423    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7\": container with ID starting with 7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7 not found: ID does not exist" containerID="7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7"
Sep 15 13:23:05.140610 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.140584    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7"} err="failed to get container status \"7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7\": rpc error: code = NotFound desc = could not find container \"7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7\": container with ID starting with 7677f587fbfa21ea1aad101d8e21683d22b6dcb1b4d4015dc113e8b6fdc154b7 not found: ID does not exist"
Sep 15 13:23:05.143155 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.143129    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"]
Sep 15 13:23:05.145420 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:05.145401    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-650c57-openshift-default-5f56df7d96-2sdxb"]
Sep 15 13:23:06.757550 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:06.757516    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cdf1993-f825-4fac-8445-e436b818319e" path="/var/lib/kubelet/pods/0cdf1993-f825-4fac-8445-e436b818319e/volumes"
Sep 15 13:23:15.115668 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.115632    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"]
Sep 15 13:23:15.116866 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.116789    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd" podUID="3c7f46b3-93ce-4355-913f-d974f590ba96" containerName="maas-api" containerID="cri-o://bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a" gracePeriod=30
Sep 15 13:23:15.375468 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.375399    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:23:15.458089 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.458054    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7f46b3-93ce-4355-913f-d974f590ba96-openshift-service-ca\") pod \"3c7f46b3-93ce-4355-913f-d974f590ba96\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") "
Sep 15 13:23:15.458089 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.458096    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-maas-api-tls\") pod \"3c7f46b3-93ce-4355-913f-d974f590ba96\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") "
Sep 15 13:23:15.458347 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.458183    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvr5l\" (UniqueName: \"kubernetes.io/projected/3c7f46b3-93ce-4355-913f-d974f590ba96-kube-api-access-nvr5l\") pod \"3c7f46b3-93ce-4355-913f-d974f590ba96\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") "
Sep 15 13:23:15.458347 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.458287    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-metrics-certs\") pod \"3c7f46b3-93ce-4355-913f-d974f590ba96\" (UID: \"3c7f46b3-93ce-4355-913f-d974f590ba96\") "
Sep 15 13:23:15.458469 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.458379    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c7f46b3-93ce-4355-913f-d974f590ba96-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c7f46b3-93ce-4355-913f-d974f590ba96" (UID: "3c7f46b3-93ce-4355-913f-d974f590ba96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:23:15.458654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.458632    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c7f46b3-93ce-4355-913f-d974f590ba96-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:15.460371 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.460340    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3c7f46b3-93ce-4355-913f-d974f590ba96" (UID: "3c7f46b3-93ce-4355-913f-d974f590ba96"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:23:15.460459 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.460371    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c7f46b3-93ce-4355-913f-d974f590ba96" (UID: "3c7f46b3-93ce-4355-913f-d974f590ba96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:23:15.460459 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.460420    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c7f46b3-93ce-4355-913f-d974f590ba96-kube-api-access-nvr5l" (OuterVolumeSpecName: "kube-api-access-nvr5l") pod "3c7f46b3-93ce-4355-913f-d974f590ba96" (UID: "3c7f46b3-93ce-4355-913f-d974f590ba96"). InnerVolumeSpecName "kube-api-access-nvr5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:23:15.560278 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.560193    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvr5l\" (UniqueName: \"kubernetes.io/projected/3c7f46b3-93ce-4355-913f-d974f590ba96-kube-api-access-nvr5l\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:15.560278 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.560276    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-metrics-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:15.560493 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:15.560289    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c7f46b3-93ce-4355-913f-d974f590ba96-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:16.180963 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.180929    2562 generic.go:356] "Generic (PLEG): container finished" podID="3c7f46b3-93ce-4355-913f-d974f590ba96" containerID="bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a" exitCode=0
Sep 15 13:23:16.181400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.180980    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"
Sep 15 13:23:16.181400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.181018    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd" event={"ID":"3c7f46b3-93ce-4355-913f-d974f590ba96","Type":"ContainerDied","Data":"bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a"}
Sep 15 13:23:16.181400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.181099    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd" event={"ID":"3c7f46b3-93ce-4355-913f-d974f590ba96","Type":"ContainerDied","Data":"b51498624fedb331e4312bd4b9112c9a6b7c27e6c8ba5894c55574e0d56fb6a6"}
Sep 15 13:23:16.181400 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.181126    2562 scope.go:117] "RemoveContainer" containerID="bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a"
Sep 15 13:23:16.191342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.191320    2562 scope.go:117] "RemoveContainer" containerID="bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a"
Sep 15 13:23:16.191610 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:23:16.191590    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a\": container with ID starting with bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a not found: ID does not exist" containerID="bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a"
Sep 15 13:23:16.191659 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.191620    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a"} err="failed to get container status \"bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a\": rpc error: code = NotFound desc = could not find container \"bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a\": container with ID starting with bd1f39ded022cf6296d82018977b2ec3c310d0ff8f549a858541f8421fda931a not found: ID does not exist"
Sep 15 13:23:16.203472 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.203445    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"]
Sep 15 13:23:16.209267 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.209244    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-92f861-6b459cfd5f-hl6zd"]
Sep 15 13:23:16.756935 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:16.756901    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c7f46b3-93ce-4355-913f-d974f590ba96" path="/var/lib/kubelet/pods/3c7f46b3-93ce-4355-913f-d974f590ba96/volumes"
Sep 15 13:23:20.691492 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:20.691455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:23:22.743335 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:22.743297    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"]
Sep 15 13:23:22.743780 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:22.743560    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy" containerID="cri-o://9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693" gracePeriod=30
Sep 15 13:23:28.010526 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.010464    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:23:28.099645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099615    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-socket\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.099830 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099663    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0632853-bc89-47ed-9622-9585abd8b5da-istiod-ca-cert\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.099830 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099681    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-credential-socket\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.099830 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099705    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sph59\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-kube-api-access-sph59\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.099830 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099743    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-certs\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.099830 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099771    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-istio-token\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.100139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099831    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0632853-bc89-47ed-9622-9585abd8b5da-istio-podinfo\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.100139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099889    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:28.100139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099909    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-data\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.100139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.099991    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:28.100139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100060    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-envoy\") pod \"f0632853-bc89-47ed-9622-9585abd8b5da\" (UID: \"f0632853-bc89-47ed-9622-9585abd8b5da\") "
Sep 15 13:23:28.100139 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100101    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:28.100402 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100150    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0632853-bc89-47ed-9622-9585abd8b5da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:23:28.100402 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100201    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:28.100528 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100509    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.100588 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100533    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0632853-bc89-47ed-9622-9585abd8b5da-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.100588 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100547    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.100588 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100561    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.100588 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.100573    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.102127 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.102096    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-kube-api-access-sph59" (OuterVolumeSpecName: "kube-api-access-sph59") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "kube-api-access-sph59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:23:28.102234 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.102122    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:28.102234 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.102122    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:23:28.102234 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.102128    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0632853-bc89-47ed-9622-9585abd8b5da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0632853-bc89-47ed-9622-9585abd8b5da" (UID: "f0632853-bc89-47ed-9622-9585abd8b5da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:23:28.201336 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.201293    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0632853-bc89-47ed-9622-9585abd8b5da-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.201336 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.201334    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sph59\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-kube-api-access-sph59\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.201336 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.201343    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0632853-bc89-47ed-9622-9585abd8b5da-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.201336 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.201353    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0632853-bc89-47ed-9622-9585abd8b5da-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:28.253654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.253619    2562 generic.go:356] "Generic (PLEG): container finished" podID="f0632853-bc89-47ed-9622-9585abd8b5da" containerID="9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693" exitCode=0
Sep 15 13:23:28.253833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.253671    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"
Sep 15 13:23:28.253833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.253669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" event={"ID":"f0632853-bc89-47ed-9622-9585abd8b5da","Type":"ContainerDied","Data":"9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693"}
Sep 15 13:23:28.253833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.253807    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4" event={"ID":"f0632853-bc89-47ed-9622-9585abd8b5da","Type":"ContainerDied","Data":"ad5d72d45821bffb3996621558dea5572116e58545bce04103bfc01647df48be"}
Sep 15 13:23:28.253833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.253834    2562 scope.go:117] "RemoveContainer" containerID="9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693"
Sep 15 13:23:28.264116 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.264096    2562 scope.go:117] "RemoveContainer" containerID="59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac"
Sep 15 13:23:28.272648 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.272631    2562 scope.go:117] "RemoveContainer" containerID="9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693"
Sep 15 13:23:28.272911 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:23:28.272896    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693\": container with ID starting with 9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693 not found: ID does not exist" containerID="9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693"
Sep 15 13:23:28.272958 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.272920    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693"} err="failed to get container status \"9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693\": rpc error: code = NotFound desc = could not find container \"9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693\": container with ID starting with 9445173264ec50c489fefe1e0a0922e5edeb1d3360e2540d323f19f9ee85f693 not found: ID does not exist"
Sep 15 13:23:28.272958 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.272939    2562 scope.go:117] "RemoveContainer" containerID="59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac"
Sep 15 13:23:28.273185 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:23:28.273169    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac\": container with ID starting with 59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac not found: ID does not exist" containerID="59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac"
Sep 15 13:23:28.273230 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.273190    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac"} err="failed to get container status \"59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac\": rpc error: code = NotFound desc = could not find container \"59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac\": container with ID starting with 59e3d8ad7c70b3fb4bb5c45dde8d43f111cf1d9f88f6390f7e1363ba27652aac not found: ID does not exist"
Sep 15 13:23:28.278124 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.278103    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"]
Sep 15 13:23:28.281466 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.281447    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-92f861-openshift-default-68dbb766bb-8zfl4"]
Sep 15 13:23:28.758302 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:28.758273    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" path="/var/lib/kubelet/pods/f0632853-bc89-47ed-9622-9585abd8b5da/volumes"
Sep 15 13:23:40.604616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:40.604537    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:23:41.558080 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:41.558045    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"]
Sep 15 13:23:41.558290 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:41.558267    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy" containerID="cri-o://30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9" gracePeriod=30
Sep 15 13:23:46.267461 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.267417    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 13:23:46.267906 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.267516    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 13:23:46.816956 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.816932    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:23:46.902714 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.902640    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3462c55f-8088-478c-b6f1-e270b3059fd4-istiod-ca-cert\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.902714 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.902685    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-socket\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.902714 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.902710    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48pxw\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-kube-api-access-48pxw\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.902941 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.902733    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-podinfo\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.902941 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.902785    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-token\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.903062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.902951    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-credential-socket\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.903062 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.902994    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-certs\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.903178 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903062    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-data\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.903178 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903080    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:46.903178 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903100    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-envoy\") pod \"3462c55f-8088-478c-b6f1-e270b3059fd4\" (UID: \"3462c55f-8088-478c-b6f1-e270b3059fd4\") "
Sep 15 13:23:46.903178 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903131    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3462c55f-8088-478c-b6f1-e270b3059fd4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:23:46.903395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903339    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:46.903456 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903394    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-data" (OuterVolumeSpecName: "istio-data") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:46.903535 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903240    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:46.903766 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903691    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:46.903766 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903716    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:46.903766 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903731    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:46.903766 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903745    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3462c55f-8088-478c-b6f1-e270b3059fd4-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:46.903766 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.903757    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:46.905164 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.905136    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:23:46.905266 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.905192    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-token" (OuterVolumeSpecName: "istio-token") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:23:46.905266 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.905203    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-kube-api-access-48pxw" (OuterVolumeSpecName: "kube-api-access-48pxw") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "kube-api-access-48pxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:23:46.905393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:46.905375    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3462c55f-8088-478c-b6f1-e270b3059fd4" (UID: "3462c55f-8088-478c-b6f1-e270b3059fd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:23:47.004502 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.004473    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:47.004502 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.004500    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:47.004706 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.004510    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48pxw\" (UniqueName: \"kubernetes.io/projected/3462c55f-8088-478c-b6f1-e270b3059fd4-kube-api-access-48pxw\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:47.004706 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.004520    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3462c55f-8088-478c-b6f1-e270b3059fd4-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:23:47.370820 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.370788    2562 generic.go:356] "Generic (PLEG): container finished" podID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerID="30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9" exitCode=0
Sep 15 13:23:47.371322 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.370847    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"
Sep 15 13:23:47.371322 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.370873    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" event={"ID":"3462c55f-8088-478c-b6f1-e270b3059fd4","Type":"ContainerDied","Data":"30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9"}
Sep 15 13:23:47.371322 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.370931    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9" event={"ID":"3462c55f-8088-478c-b6f1-e270b3059fd4","Type":"ContainerDied","Data":"c6c55b0b4295e2152b5f88207dfd5ca078b00fd9aace638e3b1164e92a0f0b4a"}
Sep 15 13:23:47.371322 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.370955    2562 scope.go:117] "RemoveContainer" containerID="30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9"
Sep 15 13:23:47.381023 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.381001    2562 scope.go:117] "RemoveContainer" containerID="ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847"
Sep 15 13:23:47.389448 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.389431    2562 scope.go:117] "RemoveContainer" containerID="30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9"
Sep 15 13:23:47.389703 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:23:47.389684    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9\": container with ID starting with 30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9 not found: ID does not exist" containerID="30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9"
Sep 15 13:23:47.389773 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.389712    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9"} err="failed to get container status \"30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9\": rpc error: code = NotFound desc = could not find container \"30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9\": container with ID starting with 30df808a7155c68f768222b452363537e504d3ce1c5f37aface6bf4c1fb3eee9 not found: ID does not exist"
Sep 15 13:23:47.389773 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.389736    2562 scope.go:117] "RemoveContainer" containerID="ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847"
Sep 15 13:23:47.389989 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:23:47.389966    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847\": container with ID starting with ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847 not found: ID does not exist" containerID="ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847"
Sep 15 13:23:47.390072 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.389999    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847"} err="failed to get container status \"ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847\": rpc error: code = NotFound desc = could not find container \"ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847\": container with ID starting with ffb777395a7ed33012b0cad8966f7a5cee9a85d155f425e746e2cee32a828847 not found: ID does not exist"
Sep 15 13:23:47.394466 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.394444    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"]
Sep 15 13:23:47.398260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:47.398239    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f6a966-openshift-default-5c78858d4b-7tvh9"]
Sep 15 13:23:48.757073 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:48.757026    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" path="/var/lib/kubelet/pods/3462c55f-8088-478c-b6f1-e270b3059fd4/volumes"
Sep 15 13:23:52.497117 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:52.497058    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"]
Sep 15 13:23:52.499176 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:52.499154    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ddd0eb-29824635-b5qv6"]
Sep 15 13:23:52.756807 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:52.756729    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84923d91-5727-4fe7-9f4b-cfba64754e4d" path="/var/lib/kubelet/pods/84923d91-5727-4fe7-9f4b-cfba64754e4d/volumes"
Sep 15 13:23:56.993576 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:56.993548    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:23:59.061482 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:59.061430    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"]
Sep 15 13:23:59.062021 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:23:59.061718    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy" containerID="cri-o://823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a" gracePeriod=30
Sep 15 13:24:04.328446 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.328368    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:24:04.389860 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.389824    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-certs\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390070 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.389877    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-credential-socket\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390070 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.389895    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-token\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390070 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.389917    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-data\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390070 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.389934    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-podinfo\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390070 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.389963    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjsjd\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-kube-api-access-cjsjd\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390070 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.389999    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istiod-ca-cert\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390404 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390110    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-socket\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390404 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390155    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:24:04.390404 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390198    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-envoy\") pod \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\" (UID: \"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8\") "
Sep 15 13:24:04.390404 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390205    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:24:04.390404 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390349    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:24:04.390665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390401    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:24:04.390665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390459    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:24:04.390665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390628    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.390665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390648    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.390665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390659    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.390665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390667    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.390665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.390675    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.392404 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.392378    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:24:04.392532 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.392459    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:24:04.392765 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.392743    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-kube-api-access-cjsjd" (OuterVolumeSpecName: "kube-api-access-cjsjd") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "kube-api-access-cjsjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:24:04.392810 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.392745    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" (UID: "a4278035-22b6-4d22-8ee3-5ee6cdc19ab8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:24:04.472199 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.472162    2562 generic.go:356] "Generic (PLEG): container finished" podID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerID="823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a" exitCode=0
Sep 15 13:24:04.472342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.472211    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"
Sep 15 13:24:04.472342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.472238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" event={"ID":"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8","Type":"ContainerDied","Data":"823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a"}
Sep 15 13:24:04.472342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.472300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr" event={"ID":"a4278035-22b6-4d22-8ee3-5ee6cdc19ab8","Type":"ContainerDied","Data":"881664af0af41a7857035e3c3adcf3fc616c2786d7a6b265877fc229a0ec8104"}
Sep 15 13:24:04.472342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.472331    2562 scope.go:117] "RemoveContainer" containerID="823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a"
Sep 15 13:24:04.486296 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.486275    2562 scope.go:117] "RemoveContainer" containerID="4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16"
Sep 15 13:24:04.491698 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.491676    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjsjd\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-kube-api-access-cjsjd\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.491778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.491699    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.491778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.491709    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.491778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.491717    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:24:04.494676 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.494662    2562 scope.go:117] "RemoveContainer" containerID="823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a"
Sep 15 13:24:04.494946 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:24:04.494926    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a\": container with ID starting with 823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a not found: ID does not exist" containerID="823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a"
Sep 15 13:24:04.494999 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.494953    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a"} err="failed to get container status \"823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a\": rpc error: code = NotFound desc = could not find container \"823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a\": container with ID starting with 823fd240c9a167d51a533c1677b71efca83b2cbe8ddf4d45de937d03872dd41a not found: ID does not exist"
Sep 15 13:24:04.494999 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.494970    2562 scope.go:117] "RemoveContainer" containerID="4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16"
Sep 15 13:24:04.495258 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:24:04.495241    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16\": container with ID starting with 4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16 not found: ID does not exist" containerID="4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16"
Sep 15 13:24:04.495318 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.495267    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16"} err="failed to get container status \"4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16\": rpc error: code = NotFound desc = could not find container \"4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16\": container with ID starting with 4a95a6003b8f94d5c1f1656afdf32c23d45f1e553e956abd1350afb629dffc16 not found: ID does not exist"
Sep 15 13:24:04.499401 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.499379    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"]
Sep 15 13:24:04.504461 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.504439    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ddd0eb-openshift-default-798b5b86b6-s94xr"]
Sep 15 13:24:04.756940 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:04.756906    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" path="/var/lib/kubelet/pods/a4278035-22b6-4d22-8ee3-5ee6cdc19ab8/volumes"
Sep 15 13:24:42.949581 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:42.949554    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:24:42.953362 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:42.953342    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:24:42.955341 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:42.955318    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:24:42.959335 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:42.959317    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:24:43.219260 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:24:43.219185    2562 scope.go:117] "RemoveContainer" containerID="06e94205d8f2188b2b75b8441cd638b0e4de4ab9598d606b5772a581a606e01f"
Sep 15 13:29:42.989530 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:29:42.989449    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:29:42.993318 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:29:42.993296    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:29:42.995436 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:29:42.995416    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:29:42.999123 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:29:42.999099    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:30:00.158393 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.158337    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"]
Sep 15 13:30:00.159094 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159064    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy"
Sep 15 13:30:00.159094 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159085    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy"
Sep 15 13:30:00.159094 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159096    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7f46b3-93ce-4355-913f-d974f590ba96" containerName="maas-api"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159105    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7f46b3-93ce-4355-913f-d974f590ba96" containerName="maas-api"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159115    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159124    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159136    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159141    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159157    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159163    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159180    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159188    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159202    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26bef37e-4751-43b8-ad13-4d18222ce8ba" containerName="maas-api"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159208    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="26bef37e-4751-43b8-ad13-4d18222ce8ba" containerName="maas-api"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159221    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159226    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159231    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159236    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159336    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy"
Sep 15 13:30:00.159345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159351    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy"
Sep 15 13:30:00.159913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159361    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy"
Sep 15 13:30:00.159913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159370    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="26bef37e-4751-43b8-ad13-4d18222ce8ba" containerName="maas-api"
Sep 15 13:30:00.159913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159376    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy"
Sep 15 13:30:00.159913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159384    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cdf1993-f825-4fac-8445-e436b818319e" containerName="istio-proxy"
Sep 15 13:30:00.159913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159391    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy"
Sep 15 13:30:00.159913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159399    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0632853-bc89-47ed-9622-9585abd8b5da" containerName="istio-proxy"
Sep 15 13:30:00.159913 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.159406    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c7f46b3-93ce-4355-913f-d974f590ba96" containerName="maas-api"
Sep 15 13:30:00.162956 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.162935    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"]
Sep 15 13:30:00.163082 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.163059    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"
Sep 15 13:30:00.163420 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.163403    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy"
Sep 15 13:30:00.163490 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.163423    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4278035-22b6-4d22-8ee3-5ee6cdc19ab8" containerName="istio-proxy"
Sep 15 13:30:00.163533 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.163522    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3462c55f-8088-478c-b6f1-e270b3059fd4" containerName="istio-proxy"
Sep 15 13:30:00.166026 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.166005    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:30:00.166748 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.166730    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"
Sep 15 13:30:00.193610 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.193584    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"]
Sep 15 13:30:00.197324 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.197299    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"
Sep 15 13:30:00.200511 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.200474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"]
Sep 15 13:30:00.203547 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.203526    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"]
Sep 15 13:30:00.217566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.217537    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"]
Sep 15 13:30:00.315460 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.315424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85rkk\" (UniqueName: \"kubernetes.io/projected/c106eab1-36fa-4907-812c-74a7a913108e-kube-api-access-85rkk\") pod \"maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5\" (UID: \"c106eab1-36fa-4907-812c-74a7a913108e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"
Sep 15 13:30:00.315650 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.315528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8496\" (UniqueName: \"kubernetes.io/projected/fd0e4617-9d38-4f82-8acd-e0426876461f-kube-api-access-v8496\") pod \"maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs\" (UID: \"fd0e4617-9d38-4f82-8acd-e0426876461f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"
Sep 15 13:30:00.315694 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.315662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n54k\" (UniqueName: \"kubernetes.io/projected/cea8328e-e069-4780-b346-fd1ff26cc4e2-kube-api-access-6n54k\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4\" (UID: \"cea8328e-e069-4780-b346-fd1ff26cc4e2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"
Sep 15 13:30:00.416794 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.416691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n54k\" (UniqueName: \"kubernetes.io/projected/cea8328e-e069-4780-b346-fd1ff26cc4e2-kube-api-access-6n54k\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4\" (UID: \"cea8328e-e069-4780-b346-fd1ff26cc4e2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"
Sep 15 13:30:00.416974 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.416847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rkk\" (UniqueName: \"kubernetes.io/projected/c106eab1-36fa-4907-812c-74a7a913108e-kube-api-access-85rkk\") pod \"maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5\" (UID: \"c106eab1-36fa-4907-812c-74a7a913108e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"
Sep 15 13:30:00.416974 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.416956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8496\" (UniqueName: \"kubernetes.io/projected/fd0e4617-9d38-4f82-8acd-e0426876461f-kube-api-access-v8496\") pod \"maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs\" (UID: \"fd0e4617-9d38-4f82-8acd-e0426876461f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"
Sep 15 13:30:00.429362 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.429331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n54k\" (UniqueName: \"kubernetes.io/projected/cea8328e-e069-4780-b346-fd1ff26cc4e2-kube-api-access-6n54k\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4\" (UID: \"cea8328e-e069-4780-b346-fd1ff26cc4e2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"
Sep 15 13:30:00.429502 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.429446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85rkk\" (UniqueName: \"kubernetes.io/projected/c106eab1-36fa-4907-812c-74a7a913108e-kube-api-access-85rkk\") pod \"maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5\" (UID: \"c106eab1-36fa-4907-812c-74a7a913108e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"
Sep 15 13:30:00.429546 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.429518    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8496\" (UniqueName: \"kubernetes.io/projected/fd0e4617-9d38-4f82-8acd-e0426876461f-kube-api-access-v8496\") pod \"maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs\" (UID: \"fd0e4617-9d38-4f82-8acd-e0426876461f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"
Sep 15 13:30:00.474433 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.474402    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"
Sep 15 13:30:00.481498 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.481473    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"
Sep 15 13:30:00.510147 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.510106    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"
Sep 15 13:30:00.757678 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:30:00.757652    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0e4617_9d38_4f82_8acd_e0426876461f.slice/crio-65455e332e4b7faf2d17a0698522282bec18781a4a003ef69819643891bfd0ce WatchSource:0}: Error finding container 65455e332e4b7faf2d17a0698522282bec18781a4a003ef69819643891bfd0ce: Status 404 returned error can't find the container with id 65455e332e4b7faf2d17a0698522282bec18781a4a003ef69819643891bfd0ce
Sep 15 13:30:00.758104 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.758084    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"]
Sep 15 13:30:00.762697 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.762677    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"]
Sep 15 13:30:00.765009 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:30:00.764982    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea8328e_e069_4780_b346_fd1ff26cc4e2.slice/crio-1000d3134c30f8b6eef11da4ca65fa7f27cbf2b61c830a7a50b0904070d460b8 WatchSource:0}: Error finding container 1000d3134c30f8b6eef11da4ca65fa7f27cbf2b61c830a7a50b0904070d460b8: Status 404 returned error can't find the container with id 1000d3134c30f8b6eef11da4ca65fa7f27cbf2b61c830a7a50b0904070d460b8
Sep 15 13:30:00.804406 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:00.804385    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"]
Sep 15 13:30:00.806293 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:30:00.806265    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc106eab1_36fa_4907_812c_74a7a913108e.slice/crio-d4e1fee2f4946961b5381fc051ad3e05e29c4f23c20ce70e986583dfec34779d WatchSource:0}: Error finding container d4e1fee2f4946961b5381fc051ad3e05e29c4f23c20ce70e986583dfec34779d: Status 404 returned error can't find the container with id d4e1fee2f4946961b5381fc051ad3e05e29c4f23c20ce70e986583dfec34779d
Sep 15 13:30:01.434568 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.434528    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5" event={"ID":"c106eab1-36fa-4907-812c-74a7a913108e","Type":"ContainerStarted","Data":"c203aecbf7476b4a960e7a5b3edac3a4a7b4734fb18cd6070cb791537429cc74"}
Sep 15 13:30:01.434984 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.434597    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5" event={"ID":"c106eab1-36fa-4907-812c-74a7a913108e","Type":"ContainerStarted","Data":"d4e1fee2f4946961b5381fc051ad3e05e29c4f23c20ce70e986583dfec34779d"}
Sep 15 13:30:01.436131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.436099    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4" event={"ID":"cea8328e-e069-4780-b346-fd1ff26cc4e2","Type":"ContainerStarted","Data":"8006693fb16520610f4cc38106b9e8edd4e3bc37fe0e319f3673a7ea3dd71c2c"}
Sep 15 13:30:01.436254 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.436156    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4" event={"ID":"cea8328e-e069-4780-b346-fd1ff26cc4e2","Type":"ContainerStarted","Data":"1000d3134c30f8b6eef11da4ca65fa7f27cbf2b61c830a7a50b0904070d460b8"}
Sep 15 13:30:01.437752 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.437729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs" event={"ID":"fd0e4617-9d38-4f82-8acd-e0426876461f","Type":"ContainerStarted","Data":"c067acc503799014a343d3d9fc51661cebd0af161dffd995d8c812f55126c486"}
Sep 15 13:30:01.437859 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.437771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs" event={"ID":"fd0e4617-9d38-4f82-8acd-e0426876461f","Type":"ContainerStarted","Data":"65455e332e4b7faf2d17a0698522282bec18781a4a003ef69819643891bfd0ce"}
Sep 15 13:30:01.454919 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.454876    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5" podStartSLOduration=1.454859963 podStartE2EDuration="1.454859963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:30:01.451740566 +0000 UTC m=+2719.244442297" watchObservedRunningTime="2026-09-15 13:30:01.454859963 +0000 UTC m=+2719.247561688"
Sep 15 13:30:01.470025 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.469980    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs" podStartSLOduration=1.4699679350000001 podStartE2EDuration="1.469967935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:30:01.467959998 +0000 UTC m=+2719.260661721" watchObservedRunningTime="2026-09-15 13:30:01.469967935 +0000 UTC m=+2719.262669665"
Sep 15 13:30:01.486356 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:01.486312    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4" podStartSLOduration=1.486298459 podStartE2EDuration="1.486298459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:30:01.484939749 +0000 UTC m=+2719.277641481" watchObservedRunningTime="2026-09-15 13:30:01.486298459 +0000 UTC m=+2719.279000190"
Sep 15 13:30:02.445813 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:02.445765    2562 generic.go:356] "Generic (PLEG): container finished" podID="c106eab1-36fa-4907-812c-74a7a913108e" containerID="c203aecbf7476b4a960e7a5b3edac3a4a7b4734fb18cd6070cb791537429cc74" exitCode=0
Sep 15 13:30:02.446235 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:02.445853    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5" event={"ID":"c106eab1-36fa-4907-812c-74a7a913108e","Type":"ContainerDied","Data":"c203aecbf7476b4a960e7a5b3edac3a4a7b4734fb18cd6070cb791537429cc74"}
Sep 15 13:30:02.447419 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:02.447397    2562 generic.go:356] "Generic (PLEG): container finished" podID="cea8328e-e069-4780-b346-fd1ff26cc4e2" containerID="8006693fb16520610f4cc38106b9e8edd4e3bc37fe0e319f3673a7ea3dd71c2c" exitCode=0
Sep 15 13:30:02.447532 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:02.447421    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4" event={"ID":"cea8328e-e069-4780-b346-fd1ff26cc4e2","Type":"ContainerDied","Data":"8006693fb16520610f4cc38106b9e8edd4e3bc37fe0e319f3673a7ea3dd71c2c"}
Sep 15 13:30:02.448867 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:02.448848    2562 generic.go:356] "Generic (PLEG): container finished" podID="fd0e4617-9d38-4f82-8acd-e0426876461f" containerID="c067acc503799014a343d3d9fc51661cebd0af161dffd995d8c812f55126c486" exitCode=0
Sep 15 13:30:02.448951 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:02.448921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs" event={"ID":"fd0e4617-9d38-4f82-8acd-e0426876461f","Type":"ContainerDied","Data":"c067acc503799014a343d3d9fc51661cebd0af161dffd995d8c812f55126c486"}
Sep 15 13:30:03.632978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.632952    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"
Sep 15 13:30:03.635978 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.635956    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"
Sep 15 13:30:03.663639 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.663617    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"
Sep 15 13:30:03.752022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.751940    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8496\" (UniqueName: \"kubernetes.io/projected/fd0e4617-9d38-4f82-8acd-e0426876461f-kube-api-access-v8496\") pod \"fd0e4617-9d38-4f82-8acd-e0426876461f\" (UID: \"fd0e4617-9d38-4f82-8acd-e0426876461f\") "
Sep 15 13:30:03.752022 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.752012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85rkk\" (UniqueName: \"kubernetes.io/projected/c106eab1-36fa-4907-812c-74a7a913108e-kube-api-access-85rkk\") pod \"c106eab1-36fa-4907-812c-74a7a913108e\" (UID: \"c106eab1-36fa-4907-812c-74a7a913108e\") "
Sep 15 13:30:03.754142 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.754118    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c106eab1-36fa-4907-812c-74a7a913108e-kube-api-access-85rkk" (OuterVolumeSpecName: "kube-api-access-85rkk") pod "c106eab1-36fa-4907-812c-74a7a913108e" (UID: "c106eab1-36fa-4907-812c-74a7a913108e"). InnerVolumeSpecName "kube-api-access-85rkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:30:03.754254 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.754229    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0e4617-9d38-4f82-8acd-e0426876461f-kube-api-access-v8496" (OuterVolumeSpecName: "kube-api-access-v8496") pod "fd0e4617-9d38-4f82-8acd-e0426876461f" (UID: "fd0e4617-9d38-4f82-8acd-e0426876461f"). InnerVolumeSpecName "kube-api-access-v8496". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:30:03.853421 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.853389    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n54k\" (UniqueName: \"kubernetes.io/projected/cea8328e-e069-4780-b346-fd1ff26cc4e2-kube-api-access-6n54k\") pod \"cea8328e-e069-4780-b346-fd1ff26cc4e2\" (UID: \"cea8328e-e069-4780-b346-fd1ff26cc4e2\") "
Sep 15 13:30:03.853827 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.853810    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85rkk\" (UniqueName: \"kubernetes.io/projected/c106eab1-36fa-4907-812c-74a7a913108e-kube-api-access-85rkk\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:30:03.853890 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.853829    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8496\" (UniqueName: \"kubernetes.io/projected/fd0e4617-9d38-4f82-8acd-e0426876461f-kube-api-access-v8496\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:30:03.855506 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.855483    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea8328e-e069-4780-b346-fd1ff26cc4e2-kube-api-access-6n54k" (OuterVolumeSpecName: "kube-api-access-6n54k") pod "cea8328e-e069-4780-b346-fd1ff26cc4e2" (UID: "cea8328e-e069-4780-b346-fd1ff26cc4e2"). InnerVolumeSpecName "kube-api-access-6n54k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:30:03.955322 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:03.955288    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n54k\" (UniqueName: \"kubernetes.io/projected/cea8328e-e069-4780-b346-fd1ff26cc4e2-kube-api-access-6n54k\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:30:04.461833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.461806    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5"
Sep 15 13:30:04.462019 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.461808    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5" event={"ID":"c106eab1-36fa-4907-812c-74a7a913108e","Type":"ContainerDied","Data":"d4e1fee2f4946961b5381fc051ad3e05e29c4f23c20ce70e986583dfec34779d"}
Sep 15 13:30:04.462019 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.461921    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4e1fee2f4946961b5381fc051ad3e05e29c4f23c20ce70e986583dfec34779d"
Sep 15 13:30:04.463250 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.463233    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"
Sep 15 13:30:04.463345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.463288    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4" event={"ID":"cea8328e-e069-4780-b346-fd1ff26cc4e2","Type":"ContainerDied","Data":"1000d3134c30f8b6eef11da4ca65fa7f27cbf2b61c830a7a50b0904070d460b8"}
Sep 15 13:30:04.463345 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.463326    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1000d3134c30f8b6eef11da4ca65fa7f27cbf2b61c830a7a50b0904070d460b8"
Sep 15 13:30:04.464657 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.464636    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs"
Sep 15 13:30:04.464771 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.464678    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs" event={"ID":"fd0e4617-9d38-4f82-8acd-e0426876461f","Type":"ContainerDied","Data":"65455e332e4b7faf2d17a0698522282bec18781a4a003ef69819643891bfd0ce"}
Sep 15 13:30:04.464771 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:30:04.464710    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65455e332e4b7faf2d17a0698522282bec18781a4a003ef69819643891bfd0ce"
Sep 15 13:32:06.758616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:06.758587    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"]
Sep 15 13:32:10.694310 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:10.694276    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:32:18.563646 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.563606    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"]
Sep 15 13:32:18.564191 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.563875    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd" podUID="55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" containerName="maas-api" containerID="cri-o://4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b" gracePeriod=30
Sep 15 13:32:18.594825 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.594796    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"]
Sep 15 13:32:18.598292 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.598251    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"]
Sep 15 13:32:18.600648 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.600624    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824650-7kkh4"]
Sep 15 13:32:18.602617 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.602596    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-aa8ef1-29824635-htj9m"]
Sep 15 13:32:18.757216 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.757186    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cea8328e-e069-4780-b346-fd1ff26cc4e2" path="/var/lib/kubelet/pods/cea8328e-e069-4780-b346-fd1ff26cc4e2/volumes"
Sep 15 13:32:18.757571 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.757552    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d392cfa0-2845-419a-81dc-6f9d8c799f6d" path="/var/lib/kubelet/pods/d392cfa0-2845-419a-81dc-6f9d8c799f6d/volumes"
Sep 15 13:32:18.814580 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:18.814529    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:32:19.003634 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.003598    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-metrics-certs\") pod \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") "
Sep 15 13:32:19.003829 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.003662    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls\") pod \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") "
Sep 15 13:32:19.003829 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.003699    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8gxw\" (UniqueName: \"kubernetes.io/projected/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-kube-api-access-t8gxw\") pod \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") "
Sep 15 13:32:19.003829 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.003726    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-openshift-service-ca\") pod \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\" (UID: \"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c\") "
Sep 15 13:32:19.004142 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.004106    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" (UID: "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:32:19.005841 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.005815    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-kube-api-access-t8gxw" (OuterVolumeSpecName: "kube-api-access-t8gxw") pod "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" (UID: "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c"). InnerVolumeSpecName "kube-api-access-t8gxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:32:19.005935 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.005845    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" (UID: "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:32:19.005935 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.005869    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" (UID: "55f5a4d5-a122-49e6-85f1-f5aa2ce1508c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:32:19.104778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.104705    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-metrics-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:19.104778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.104733    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:19.104778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.104743    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8gxw\" (UniqueName: \"kubernetes.io/projected/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-kube-api-access-t8gxw\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:19.104778 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.104751    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:19.202704 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.202672    2562 generic.go:356] "Generic (PLEG): container finished" podID="55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" containerID="4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b" exitCode=0
Sep 15 13:32:19.202858 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.202714    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"
Sep 15 13:32:19.202858 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.202749    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd" event={"ID":"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c","Type":"ContainerDied","Data":"4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b"}
Sep 15 13:32:19.202858 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.202800    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd" event={"ID":"55f5a4d5-a122-49e6-85f1-f5aa2ce1508c","Type":"ContainerDied","Data":"a964cbe9beefe5689ff2183f69cc093bf05bf9adbf90e678b64ea8695040fb47"}
Sep 15 13:32:19.202858 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.202824    2562 scope.go:117] "RemoveContainer" containerID="4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b"
Sep 15 13:32:19.212189 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.212171    2562 scope.go:117] "RemoveContainer" containerID="4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b"
Sep 15 13:32:19.212469 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:32:19.212449    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b\": container with ID starting with 4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b not found: ID does not exist" containerID="4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b"
Sep 15 13:32:19.212563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.212479    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b"} err="failed to get container status \"4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b\": rpc error: code = NotFound desc = could not find container \"4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b\": container with ID starting with 4c21b40a0cd0ab4ee48fb0da621b563bc35eb56b657dfa70d0f5efd1023c4c1b not found: ID does not exist"
Sep 15 13:32:19.229855 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.229822    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"]
Sep 15 13:32:19.240094 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:19.240068    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-aa8ef1-58fd8b6d88-587pd"]
Sep 15 13:32:20.756654 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:20.756622    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" path="/var/lib/kubelet/pods/55f5a4d5-a122-49e6-85f1-f5aa2ce1508c/volumes"
Sep 15 13:32:21.497382 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:21.497349    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:32:21.777399 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:21.777312    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt" podUID="3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" containerName="main" containerID="cri-o://4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80" gracePeriod=45
Sep 15 13:32:22.043977 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.043644    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:32:22.133364 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133331    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-home\") pod \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") "
Sep 15 13:32:22.133364 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133364    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-dshm\") pod \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") "
Sep 15 13:32:22.133616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133403    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tmp-dir\") pod \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") "
Sep 15 13:32:22.133616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133450    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tls-certs\") pod \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") "
Sep 15 13:32:22.133616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133516    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-model-cache\") pod \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") "
Sep 15 13:32:22.133616 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133550    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwgq8\" (UniqueName: \"kubernetes.io/projected/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-kube-api-access-xwgq8\") pod \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\" (UID: \"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3\") "
Sep 15 13:32:22.133846 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" (UID: "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:22.133846 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133801    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-model-cache" (OuterVolumeSpecName: "model-cache") pod "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" (UID: "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:22.133846 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133814    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-home" (OuterVolumeSpecName: "home") pod "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" (UID: "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:22.134050 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.133998    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-home\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:22.134132 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.134063    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tmp-dir\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:22.134132 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.134079    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-model-cache\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:22.135727 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.135700    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-dshm" (OuterVolumeSpecName: "dshm") pod "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" (UID: "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:22.135727 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.135707    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" (UID: "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 13:32:22.135857 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.135762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-kube-api-access-xwgq8" (OuterVolumeSpecName: "kube-api-access-xwgq8") pod "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" (UID: "3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3"). InnerVolumeSpecName "kube-api-access-xwgq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:32:22.223092 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.223024    2562 generic.go:356] "Generic (PLEG): container finished" podID="3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" containerID="4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80" exitCode=0
Sep 15 13:32:22.223092 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.223075    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"
Sep 15 13:32:22.223306 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.223063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt" event={"ID":"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3","Type":"ContainerDied","Data":"4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80"}
Sep 15 13:32:22.223306 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.223168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt" event={"ID":"3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3","Type":"ContainerDied","Data":"c415b0a30050cc07b9189bbda8d357cd821545b1e04df1dba3b98ef5528c2588"}
Sep 15 13:32:22.223306 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.223194    2562 scope.go:117] "RemoveContainer" containerID="4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80"
Sep 15 13:32:22.235903 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.234423    2562 scope.go:117] "RemoveContainer" containerID="4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80"
Sep 15 13:32:22.235903 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.235066    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwgq8\" (UniqueName: \"kubernetes.io/projected/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-kube-api-access-xwgq8\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:22.235903 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.235087    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-dshm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:22.235903 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.235104    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3-tls-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:22.236967 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:32:22.236938    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80\": container with ID starting with 4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80 not found: ID does not exist" containerID="4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80"
Sep 15 13:32:22.237116 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.236977    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80"} err="failed to get container status \"4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80\": rpc error: code = NotFound desc = could not find container \"4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80\": container with ID starting with 4f4893801caa8375e315ce354e9682b5edeb2a6a5dc69f431c6558b7e6e34a80 not found: ID does not exist"
Sep 15 13:32:22.245860 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.245838    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"]
Sep 15 13:32:22.247883 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.247858    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-aa8ef1/auth-test-model-aa8ef1-kserve-557dcfbbff-r5pmt"]
Sep 15 13:32:22.761531 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:22.761499    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" path="/var/lib/kubelet/pods/3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3/volumes"
Sep 15 13:32:25.497696 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.497661    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"]
Sep 15 13:32:25.498127 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.497929    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy" containerID="cri-o://de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d" gracePeriod=30
Sep 15 13:32:25.851089 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851007    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"]
Sep 15 13:32:25.851478 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851464    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea8328e-e069-4780-b346-fd1ff26cc4e2" containerName="cleanup"
Sep 15 13:32:25.851478 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851479    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea8328e-e069-4780-b346-fd1ff26cc4e2" containerName="cleanup"
Sep 15 13:32:25.851563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851491    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c106eab1-36fa-4907-812c-74a7a913108e" containerName="cleanup"
Sep 15 13:32:25.851563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851497    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c106eab1-36fa-4907-812c-74a7a913108e" containerName="cleanup"
Sep 15 13:32:25.851563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851507    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" containerName="main"
Sep 15 13:32:25.851563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851513    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" containerName="main"
Sep 15 13:32:25.851563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851535    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" containerName="maas-api"
Sep 15 13:32:25.851563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851540    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" containerName="maas-api"
Sep 15 13:32:25.851563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851546    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0e4617-9d38-4f82-8acd-e0426876461f" containerName="cleanup"
Sep 15 13:32:25.851563 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851551    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0e4617-9d38-4f82-8acd-e0426876461f" containerName="cleanup"
Sep 15 13:32:25.851831 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851607    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c106eab1-36fa-4907-812c-74a7a913108e" containerName="cleanup"
Sep 15 13:32:25.851831 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851615    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="55f5a4d5-a122-49e6-85f1-f5aa2ce1508c" containerName="maas-api"
Sep 15 13:32:25.851831 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851620    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea8328e-e069-4780-b346-fd1ff26cc4e2" containerName="cleanup"
Sep 15 13:32:25.851831 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851627    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fe70d49-ed1c-4a74-b7b2-9ce75abc77b3" containerName="main"
Sep 15 13:32:25.851831 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.851634    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0e4617-9d38-4f82-8acd-e0426876461f" containerName="cleanup"
Sep 15 13:32:25.855096 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.855076    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:25.857607 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.857590    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 13:32:25.857720 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.857696    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mn87z\""
Sep 15 13:32:25.857798 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.857781    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 13:32:25.863926 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.863898    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"]
Sep 15 13:32:25.972992 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.972963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd05425-f322-43fb-8665-ff11d367e472-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll\" (UID: \"ddd05425-f322-43fb-8665-ff11d367e472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:25.973162 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:25.973010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lckrh\" (UniqueName: \"kubernetes.io/projected/ddd05425-f322-43fb-8665-ff11d367e472-kube-api-access-lckrh\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll\" (UID: \"ddd05425-f322-43fb-8665-ff11d367e472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:26.074644 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:26.074613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd05425-f322-43fb-8665-ff11d367e472-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll\" (UID: \"ddd05425-f322-43fb-8665-ff11d367e472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:26.074857 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:26.074676    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lckrh\" (UniqueName: \"kubernetes.io/projected/ddd05425-f322-43fb-8665-ff11d367e472-kube-api-access-lckrh\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll\" (UID: \"ddd05425-f322-43fb-8665-ff11d367e472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:26.076021 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:26.075992    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd05425-f322-43fb-8665-ff11d367e472-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll\" (UID: \"ddd05425-f322-43fb-8665-ff11d367e472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:26.083294 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:26.083269    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lckrh\" (UniqueName: \"kubernetes.io/projected/ddd05425-f322-43fb-8665-ff11d367e472-kube-api-access-lckrh\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll\" (UID: \"ddd05425-f322-43fb-8665-ff11d367e472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:26.167243 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:26.167164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:26.439353 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:26.439325    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"]
Sep 15 13:32:26.442556 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:32:26.442533    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd05425_f322_43fb_8665_ff11d367e472.slice/crio-c6b33a18ee758c1164282e4e2b4237d08c3dfed42c7ae5d37ec9d5ad3b17ad8f WatchSource:0}: Error finding container c6b33a18ee758c1164282e4e2b4237d08c3dfed42c7ae5d37ec9d5ad3b17ad8f: Status 404 returned error can't find the container with id c6b33a18ee758c1164282e4e2b4237d08c3dfed42c7ae5d37ec9d5ad3b17ad8f
Sep 15 13:32:27.254513 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:27.254485    2562 generic.go:356] "Generic (PLEG): container finished" podID="ddd05425-f322-43fb-8665-ff11d367e472" containerID="38a13c471ffa189d147769cc584d116afa4cc17c23c1900342e4242766f249c0" exitCode=0
Sep 15 13:32:27.254815 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:27.254520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll" event={"ID":"ddd05425-f322-43fb-8665-ff11d367e472","Type":"ContainerDied","Data":"38a13c471ffa189d147769cc584d116afa4cc17c23c1900342e4242766f249c0"}
Sep 15 13:32:27.254815 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:27.254559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll" event={"ID":"ddd05425-f322-43fb-8665-ff11d367e472","Type":"ContainerStarted","Data":"c6b33a18ee758c1164282e4e2b4237d08c3dfed42c7ae5d37ec9d5ad3b17ad8f"}
Sep 15 13:32:28.398474 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:28.398450    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:28.500309 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:28.500278    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd05425-f322-43fb-8665-ff11d367e472-maas-api-service-ca\") pod \"ddd05425-f322-43fb-8665-ff11d367e472\" (UID: \"ddd05425-f322-43fb-8665-ff11d367e472\") "
Sep 15 13:32:28.500498 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:28.500398    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lckrh\" (UniqueName: \"kubernetes.io/projected/ddd05425-f322-43fb-8665-ff11d367e472-kube-api-access-lckrh\") pod \"ddd05425-f322-43fb-8665-ff11d367e472\" (UID: \"ddd05425-f322-43fb-8665-ff11d367e472\") "
Sep 15 13:32:28.500645 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:28.500622    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddd05425-f322-43fb-8665-ff11d367e472-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ddd05425-f322-43fb-8665-ff11d367e472" (UID: "ddd05425-f322-43fb-8665-ff11d367e472"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:32:28.502464 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:28.502435    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd05425-f322-43fb-8665-ff11d367e472-kube-api-access-lckrh" (OuterVolumeSpecName: "kube-api-access-lckrh") pod "ddd05425-f322-43fb-8665-ff11d367e472" (UID: "ddd05425-f322-43fb-8665-ff11d367e472"). InnerVolumeSpecName "kube-api-access-lckrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:32:28.601901 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:28.601811    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd05425-f322-43fb-8665-ff11d367e472-maas-api-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:28.601901 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:28.601839    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lckrh\" (UniqueName: \"kubernetes.io/projected/ddd05425-f322-43fb-8665-ff11d367e472-kube-api-access-lckrh\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:29.266795 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:29.266764    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"
Sep 15 13:32:29.267001 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:29.266804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll" event={"ID":"ddd05425-f322-43fb-8665-ff11d367e472","Type":"ContainerDied","Data":"c6b33a18ee758c1164282e4e2b4237d08c3dfed42c7ae5d37ec9d5ad3b17ad8f"}
Sep 15 13:32:29.267001 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:29.266852    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6b33a18ee758c1164282e4e2b4237d08c3dfed42c7ae5d37ec9d5ad3b17ad8f"
Sep 15 13:32:29.435290 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:29.435250    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 13:32:29.435752 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:29.435318    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 13:32:30.757148 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.757124    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:32:30.826665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.826590    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-certs\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.826665 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.826650    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72c243a2-0019-4f53-b591-53708d8cdef6-istiod-ca-cert\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.826885 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.826720    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-socket\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.826885 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.826756    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72c243a2-0019-4f53-b591-53708d8cdef6-istio-podinfo\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.826885 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.826797    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2hr5\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-kube-api-access-k2hr5\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.826885 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.826826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-envoy\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.827109 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.826896    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:30.827109 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827018    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:30.827109 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827095    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c243a2-0019-4f53-b591-53708d8cdef6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:32:30.827269 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827115    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-data\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.827269 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827152    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-istio-token\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.827269 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827200    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-credential-socket\") pod \"72c243a2-0019-4f53-b591-53708d8cdef6\" (UID: \"72c243a2-0019-4f53-b591-53708d8cdef6\") "
Sep 15 13:32:30.827406 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827262    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-data" (OuterVolumeSpecName: "istio-data") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:30.827501 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827471    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:30.827658 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827640    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:30.827715 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827664    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:30.827715 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827689    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:30.827715 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827704    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72c243a2-0019-4f53-b591-53708d8cdef6-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:30.827814 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.827716    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:30.829196 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.829172    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-istio-token" (OuterVolumeSpecName: "istio-token") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:32:30.829476 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.829458    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-kube-api-access-k2hr5" (OuterVolumeSpecName: "kube-api-access-k2hr5") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "kube-api-access-k2hr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:32:30.829567 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.829545    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72c243a2-0019-4f53-b591-53708d8cdef6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:32:30.829735 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.829720    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72c243a2-0019-4f53-b591-53708d8cdef6" (UID: "72c243a2-0019-4f53-b591-53708d8cdef6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:30.928411 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.928382    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72c243a2-0019-4f53-b591-53708d8cdef6-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:30.928411 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.928407    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2hr5\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-kube-api-access-k2hr5\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:30.928411 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.928417    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72c243a2-0019-4f53-b591-53708d8cdef6-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:30.928673 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:30.928426    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72c243a2-0019-4f53-b591-53708d8cdef6-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:31.280532 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.280497    2562 generic.go:356] "Generic (PLEG): container finished" podID="72c243a2-0019-4f53-b591-53708d8cdef6" containerID="de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d" exitCode=0
Sep 15 13:32:31.280732 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.280542    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"
Sep 15 13:32:31.280732 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.280585    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" event={"ID":"72c243a2-0019-4f53-b591-53708d8cdef6","Type":"ContainerDied","Data":"de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d"}
Sep 15 13:32:31.280732 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.280649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz" event={"ID":"72c243a2-0019-4f53-b591-53708d8cdef6","Type":"ContainerDied","Data":"ddd24570c130ccb8aa8260d19d9fce49855811da706bc91a222dd9c95d6cb339"}
Sep 15 13:32:31.280732 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.280683    2562 scope.go:117] "RemoveContainer" containerID="de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d"
Sep 15 13:32:31.292531 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.292172    2562 scope.go:117] "RemoveContainer" containerID="409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b"
Sep 15 13:32:31.300917 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.300899    2562 scope.go:117] "RemoveContainer" containerID="de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d"
Sep 15 13:32:31.301247 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:32:31.301227    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d\": container with ID starting with de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d not found: ID does not exist" containerID="de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d"
Sep 15 13:32:31.301332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.301255    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d"} err="failed to get container status \"de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d\": rpc error: code = NotFound desc = could not find container \"de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d\": container with ID starting with de6324389c0f0653fea17ce8504b8e720c206eaee37627c77b1ad90c265ee11d not found: ID does not exist"
Sep 15 13:32:31.301332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.301272    2562 scope.go:117] "RemoveContainer" containerID="409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b"
Sep 15 13:32:31.301527 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:32:31.301511    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b\": container with ID starting with 409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b not found: ID does not exist" containerID="409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b"
Sep 15 13:32:31.301568 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.301534    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b"} err="failed to get container status \"409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b\": rpc error: code = NotFound desc = could not find container \"409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b\": container with ID starting with 409765fc59918a569d6dcee5712af3ef16ba66b043d080826f96c729b4db0a5b not found: ID does not exist"
Sep 15 13:32:31.305725 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.305694    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"]
Sep 15 13:32:31.312272 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:31.312251    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa8ef1-openshift-default-84f75d4f44-mffcz"]
Sep 15 13:32:32.756960 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:32.756922    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" path="/var/lib/kubelet/pods/72c243a2-0019-4f53-b591-53708d8cdef6/volumes"
Sep 15 13:32:36.546831 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:36.546753    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"]
Sep 15 13:32:36.549603 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:36.549576    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-38d90f-6glll"]
Sep 15 13:32:36.756073 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:36.756019    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddd05425-f322-43fb-8665-ff11d367e472" path="/var/lib/kubelet/pods/ddd05425-f322-43fb-8665-ff11d367e472/volumes"
Sep 15 13:32:39.097198 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:39.097164    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:32:43.306246 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:43.306212    2562 scope.go:117] "RemoveContainer" containerID="1b0e4731ba9bfcb2ac1a8662f550cb4f8ae877ba65ee53db4e954f61e4d55c0c"
Sep 15 13:32:43.593114 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:43.593017    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"]
Sep 15 13:32:43.593284 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:43.593236    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy" containerID="cri-o://8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558" gracePeriod=30
Sep 15 13:32:48.836384 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:48.836355    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:32:49.012740 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012712    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-certs\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.012928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012748    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-credential-socket\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.012928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012767    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-envoy\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.012928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012804    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-socket\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.012928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012830    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw7wp\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-kube-api-access-kw7wp\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.012928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012851    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-podinfo\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.012928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012870    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istiod-ca-cert\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.012928 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012893    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-token\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.013342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.012967    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-data\") pod \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\" (UID: \"3f6e1bad-60f7-43a4-b48d-aa4965c057c2\") "
Sep 15 13:32:49.013342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.013073    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:49.013342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.013088    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:49.013342 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.013179    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:49.013566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.013348    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:49.013566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.013381    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 13:32:49.013566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.013418    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.013566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.013450    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-credential-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.013566 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.013469    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-workload-socket\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.015040 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.015007    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 13:32:49.015221 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.015202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:32:49.015473 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.015448    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 13:32:49.015567 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.015490    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-kube-api-access-kw7wp" (OuterVolumeSpecName: "kube-api-access-kw7wp") pod "3f6e1bad-60f7-43a4-b48d-aa4965c057c2" (UID: "3f6e1bad-60f7-43a4-b48d-aa4965c057c2"). InnerVolumeSpecName "kube-api-access-kw7wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 13:32:49.114134 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.114107    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-envoy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.114134 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.114134    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw7wp\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-kube-api-access-kw7wp\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.114332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.114149    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-podinfo\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.114332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.114162    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istiod-ca-cert\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.114332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.114173    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-token\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.114332 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.114183    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f6e1bad-60f7-43a4-b48d-aa4965c057c2-istio-data\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\""
Sep 15 13:32:49.392647 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.392567    2562 generic.go:356] "Generic (PLEG): container finished" podID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerID="8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558" exitCode=0
Sep 15 13:32:49.392647 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.392614    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"
Sep 15 13:32:49.392825 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.392658    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" event={"ID":"3f6e1bad-60f7-43a4-b48d-aa4965c057c2","Type":"ContainerDied","Data":"8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558"}
Sep 15 13:32:49.392825 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.392714    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x" event={"ID":"3f6e1bad-60f7-43a4-b48d-aa4965c057c2","Type":"ContainerDied","Data":"795488e830d8179b3909f2e46558d490703d0ab41eed6b6aaf950dd8e4e5eda3"}
Sep 15 13:32:49.392825 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.392739    2562 scope.go:117] "RemoveContainer" containerID="8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558"
Sep 15 13:32:49.402736 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.402720    2562 scope.go:117] "RemoveContainer" containerID="f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f"
Sep 15 13:32:49.410779 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.410763    2562 scope.go:117] "RemoveContainer" containerID="8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558"
Sep 15 13:32:49.411020 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:32:49.411001    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558\": container with ID starting with 8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558 not found: ID does not exist" containerID="8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558"
Sep 15 13:32:49.411116 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.411039    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558"} err="failed to get container status \"8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558\": rpc error: code = NotFound desc = could not find container \"8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558\": container with ID starting with 8e03b1e819a03686e564e750245639389d7b200b5929760119871a78b4489558 not found: ID does not exist"
Sep 15 13:32:49.411116 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.411058    2562 scope.go:117] "RemoveContainer" containerID="f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f"
Sep 15 13:32:49.411292 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:32:49.411276    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f\": container with ID starting with f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f not found: ID does not exist" containerID="f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f"
Sep 15 13:32:49.411333 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.411296    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f"} err="failed to get container status \"f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f\": rpc error: code = NotFound desc = could not find container \"f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f\": container with ID starting with f2e53056bf6f8f34651364efd24f1bc6b3aece3fd84d1a8ffec39a8ed283928f not found: ID does not exist"
Sep 15 13:32:49.414658 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.414637    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"]
Sep 15 13:32:49.419069 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:49.419049    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-38d90f-openshift-default-789c988678-jwv7x"]
Sep 15 13:32:50.756618 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:32:50.756582    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" path="/var/lib/kubelet/pods/3f6e1bad-60f7-43a4-b48d-aa4965c057c2/volumes"
Sep 15 13:34:22.441196 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441161    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f4dcdf76b-6t6qw"]
Sep 15 13:34:22.441666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441556    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd05425-f322-43fb-8665-ff11d367e472" containerName="revoke-keys"
Sep 15 13:34:22.441666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441567    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd05425-f322-43fb-8665-ff11d367e472" containerName="revoke-keys"
Sep 15 13:34:22.441666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441581    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy"
Sep 15 13:34:22.441666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441587    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy"
Sep 15 13:34:22.441666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441597    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy"
Sep 15 13:34:22.441666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy"
Sep 15 13:34:22.441666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441612    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy"
Sep 15 13:34:22.441666 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441619    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy"
Sep 15 13:34:22.441929 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441690    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy"
Sep 15 13:34:22.441929 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441699    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy"
Sep 15 13:34:22.441929 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.441716    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd05425-f322-43fb-8665-ff11d367e472" containerName="revoke-keys"
Sep 15 13:34:22.445521 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.445503    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.448498 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.448476    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 13:34:22.448498 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.448484    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-s85lm\""
Sep 15 13:34:22.448498 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.448495    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 13:34:22.448727 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.448486    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 13:34:22.455880 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.455860    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f4dcdf76b-6t6qw"]
Sep 15 13:34:22.490300 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.490277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nhv6\" (UniqueName: \"kubernetes.io/projected/81a93706-a3b7-435c-a6a0-7ebb65663aa9-kube-api-access-9nhv6\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.490422 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.490342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81a93706-a3b7-435c-a6a0-7ebb65663aa9-webhook-cert\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.490483 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.490465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a93706-a3b7-435c-a6a0-7ebb65663aa9-metrics-certs\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.591544 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.591513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a93706-a3b7-435c-a6a0-7ebb65663aa9-metrics-certs\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.591707 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.591555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhv6\" (UniqueName: \"kubernetes.io/projected/81a93706-a3b7-435c-a6a0-7ebb65663aa9-kube-api-access-9nhv6\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.591850 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.591737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81a93706-a3b7-435c-a6a0-7ebb65663aa9-webhook-cert\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.594083 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.594061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81a93706-a3b7-435c-a6a0-7ebb65663aa9-webhook-cert\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.594172 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.594119    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a93706-a3b7-435c-a6a0-7ebb65663aa9-metrics-certs\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.600473 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.600452    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhv6\" (UniqueName: \"kubernetes.io/projected/81a93706-a3b7-435c-a6a0-7ebb65663aa9-kube-api-access-9nhv6\") pod \"maas-controller-5f4dcdf76b-6t6qw\" (UID: \"81a93706-a3b7-435c-a6a0-7ebb65663aa9\") " pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:22.756965 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:22.756884    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:23.029451 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:23.029428    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f4dcdf76b-6t6qw"]
Sep 15 13:34:23.031021 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:34:23.030982    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a93706_a3b7_435c_a6a0_7ebb65663aa9.slice/crio-4a0cc176df0218df97bb80eee858fd6be9a3658369a937499a06ca5e7b9ab60b WatchSource:0}: Error finding container 4a0cc176df0218df97bb80eee858fd6be9a3658369a937499a06ca5e7b9ab60b: Status 404 returned error can't find the container with id 4a0cc176df0218df97bb80eee858fd6be9a3658369a937499a06ca5e7b9ab60b
Sep 15 13:34:23.032343 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:23.032326    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 13:34:23.906489 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:23.906441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" event={"ID":"81a93706-a3b7-435c-a6a0-7ebb65663aa9","Type":"ContainerStarted","Data":"4a0cc176df0218df97bb80eee858fd6be9a3658369a937499a06ca5e7b9ab60b"}
Sep 15 13:34:26.926414 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:26.926372    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" event={"ID":"81a93706-a3b7-435c-a6a0-7ebb65663aa9","Type":"ContainerStarted","Data":"5afa6e434c03061c62c6e1320d69e6749f5fc2f2c73560534c3d03d2fab40507"}
Sep 15 13:34:26.926788 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:26.926539    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:26.945703 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:26.945655    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" podStartSLOduration=2.080344081 podStartE2EDuration="4.945639418s" totalImagesPullingTime="2.865295337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:34:26.943378167 +0000 UTC m=+2984.736079899" watchObservedRunningTime="2026-09-15 13:34:26.945639418 +0000 UTC m=+2984.738341148"
Sep 15 13:34:37.934919 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:37.934888    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:34:43.025314 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:43.025211    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:34:43.032198 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:43.029188    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:34:43.033513 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:43.033492    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:34:43.037056 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:34:43.037019    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:36:43.372627 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:36:43.372586    2562 scope.go:117] "RemoveContainer" containerID="8006693fb16520610f4cc38106b9e8edd4e3bc37fe0e319f3673a7ea3dd71c2c"
Sep 15 13:36:45.676954 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:36:45.676917    2562 generic.go:356] "Generic (PLEG): container finished" podID="81a93706-a3b7-435c-a6a0-7ebb65663aa9" containerID="5afa6e434c03061c62c6e1320d69e6749f5fc2f2c73560534c3d03d2fab40507" exitCode=1
Sep 15 13:36:45.677385 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:36:45.676990    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" event={"ID":"81a93706-a3b7-435c-a6a0-7ebb65663aa9","Type":"ContainerDied","Data":"5afa6e434c03061c62c6e1320d69e6749f5fc2f2c73560534c3d03d2fab40507"}
Sep 15 13:36:45.677385 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:36:45.677363    2562 scope.go:117] "RemoveContainer" containerID="5afa6e434c03061c62c6e1320d69e6749f5fc2f2c73560534c3d03d2fab40507"
Sep 15 13:36:46.684203 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:36:46.684170    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" event={"ID":"81a93706-a3b7-435c-a6a0-7ebb65663aa9","Type":"ContainerStarted","Data":"05852f861d7e7f8004ace3d01bbade28eccdaab251a8aee509e2f187439f38c8"}
Sep 15 13:36:46.684580 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:36:46.684372    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:36:57.692838 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:36:57.692805    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:38:43.401271 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:38:43.401191    2562 scope.go:117] "RemoveContainer" containerID="38a13c471ffa189d147769cc584d116afa4cc17c23c1900342e4242766f249c0"
Sep 15 13:39:02.406491 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:02.406463    2562 generic.go:356] "Generic (PLEG): container finished" podID="81a93706-a3b7-435c-a6a0-7ebb65663aa9" containerID="05852f861d7e7f8004ace3d01bbade28eccdaab251a8aee509e2f187439f38c8" exitCode=1
Sep 15 13:39:02.406804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:02.406534    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" event={"ID":"81a93706-a3b7-435c-a6a0-7ebb65663aa9","Type":"ContainerDied","Data":"05852f861d7e7f8004ace3d01bbade28eccdaab251a8aee509e2f187439f38c8"}
Sep 15 13:39:02.406804 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:02.406595    2562 scope.go:117] "RemoveContainer" containerID="5afa6e434c03061c62c6e1320d69e6749f5fc2f2c73560534c3d03d2fab40507"
Sep 15 13:39:02.406921 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:02.406880    2562 scope.go:117] "RemoveContainer" containerID="05852f861d7e7f8004ace3d01bbade28eccdaab251a8aee509e2f187439f38c8"
Sep 15 13:39:02.407128 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:39:02.407110    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-5f4dcdf76b-6t6qw_opendatahub(81a93706-a3b7-435c-a6a0-7ebb65663aa9)\"" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" podUID="81a93706-a3b7-435c-a6a0-7ebb65663aa9"
Sep 15 13:39:02.757564 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:02.757532    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:39:03.413498 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:03.413470    2562 scope.go:117] "RemoveContainer" containerID="05852f861d7e7f8004ace3d01bbade28eccdaab251a8aee509e2f187439f38c8"
Sep 15 13:39:03.413890 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:39:03.413636    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-5f4dcdf76b-6t6qw_opendatahub(81a93706-a3b7-435c-a6a0-7ebb65663aa9)\"" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" podUID="81a93706-a3b7-435c-a6a0-7ebb65663aa9"
Sep 15 13:39:07.690749 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:07.690712    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:39:07.691158 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:07.691104    2562 scope.go:117] "RemoveContainer" containerID="05852f861d7e7f8004ace3d01bbade28eccdaab251a8aee509e2f187439f38c8"
Sep 15 13:39:07.691333 ip-10-0-132-169 kubenswrapper[2562]: E0915 13:39:07.691311    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-5f4dcdf76b-6t6qw_opendatahub(81a93706-a3b7-435c-a6a0-7ebb65663aa9)\"" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" podUID="81a93706-a3b7-435c-a6a0-7ebb65663aa9"
Sep 15 13:39:22.752526 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:22.752495    2562 scope.go:117] "RemoveContainer" containerID="05852f861d7e7f8004ace3d01bbade28eccdaab251a8aee509e2f187439f38c8"
Sep 15 13:39:23.520024 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:23.519988    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw" event={"ID":"81a93706-a3b7-435c-a6a0-7ebb65663aa9","Type":"ContainerStarted","Data":"bf65470a4d98ccb540af1323d0b20ae5bcf800a8cc0174ea5bd52f4b62b804aa"}
Sep 15 13:39:23.520241 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:23.520224    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:39:30.925587 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:30.925551    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gx4hh_91586217-bf10-4574-a510-9787484c3360/manager/0.log"
Sep 15 13:39:31.039357 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:31.039311    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-flzms_98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e/manager/0.log"
Sep 15 13:39:31.146313 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:31.146283    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f4dcdf76b-6t6qw_81a93706-a3b7-435c-a6a0-7ebb65663aa9/manager/2.log"
Sep 15 13:39:31.371973 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:31.371890    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2pns4_51f724a6-cbd4-47a3-86e7-ef5565c01ec3/manager/0.log"
Sep 15 13:39:31.480982 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:31.480952    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79946fddd6-l9vd4_653dfa3e-9221-4267-b320-e23bfe775222/copy-manifests/0.log"
Sep 15 13:39:31.493994 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:31.493954    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79946fddd6-l9vd4_653dfa3e-9221-4267-b320-e23bfe775222/manager/0.log"
Sep 15 13:39:33.016290 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:33.016256    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-inf-a-528f6b-29824620-rwdxc_a744131f-9791-45b8-9815-537ab9b389ae/cleanup/0.log"
Sep 15 13:39:33.235941 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:33.235910    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-inf-a-528f6b-29824650-mgzd5_c106eab1-36fa-4907-812c-74a7a913108e/cleanup/0.log"
Sep 15 13:39:33.670854 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:33.670824    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824620-hdlgv_35b2317b-dc8a-4d60-9ec6-9621e2102196/cleanup/0.log"
Sep 15 13:39:33.780193 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:33.780162    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-worker-w0-6c9f0b-29824635-htw7b_aeeecd6f-d6dc-4b5d-8d2d-99da5a4dd092/cleanup/0.log"
Sep 15 13:39:34.222736 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:34.222709    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-worker-w4-2af9a3-29824650-k7qcs_fd0e4617-9d38-4f82-8acd-e0426876461f/cleanup/0.log"
Sep 15 13:39:34.527798 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:34.527711    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f4dcdf76b-6t6qw"
Sep 15 13:39:35.332992 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:35.332949    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-4grfh_48da0f6b-419d-4bef-841a-25ef4899e8bf/manager/0.log"
Sep 15 13:39:35.554896 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:35.554854    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-9mx7j_d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5/manager/0.log"
Sep 15 13:39:35.660868 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:35.660780    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-92knm_2de15519-4f94-4f66-abb2-1fead0097a15/limitador/0.log"
Sep 15 13:39:36.129906 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:36.129875    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78_7a531882-df52-4750-a943-2153d5ae86f9/istio-proxy/0.log"
Sep 15 13:39:36.575243 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:36.575209    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8jxq4_d584efee-f195-4b32-90a9-a158bb292b71/istio-proxy/1.log"
Sep 15 13:39:37.798085 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:37.798052    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-89fbc77c4-kzvnf_7ee18f00-83f0-4177-906f-f66a46579c82/router/0.log"
Sep 15 13:39:38.266054 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:38.265996    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mxtjk_034b91e5-25f7-4b7b-a30b-1e5d108ba40d/main/0.log"
Sep 15 13:39:38.593422 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:38.593342    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp9zc4_92ecc810-f364-4665-8791-9d42635c59cb/main/0.log"
Sep 15 13:39:43.060925 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.060800    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:39:43.074539 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.071772    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:39:43.077461 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.077441    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:39:43.081455 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.081430    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7qpxx_b6a65b2e-c744-46f4-b9ca-b7166c2f9107/ovn-acl-logging/0.log"
Sep 15 13:39:43.855366 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.855331    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xpvzq/must-gather-zz546"]
Sep 15 13:39:43.855810 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.855796    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy"
Sep 15 13:39:43.855855 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.855812    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy"
Sep 15 13:39:43.855906 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.855892    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f6e1bad-60f7-43a4-b48d-aa4965c057c2" containerName="istio-proxy"
Sep 15 13:39:43.855942 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.855911    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="72c243a2-0019-4f53-b591-53708d8cdef6" containerName="istio-proxy"
Sep 15 13:39:43.859237 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.859216    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpvzq/must-gather-zz546"
Sep 15 13:39:43.861784 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.861760    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpvzq\"/\"kube-root-ca.crt\""
Sep 15 13:39:43.861939 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.861920    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpvzq\"/\"openshift-service-ca.crt\""
Sep 15 13:39:43.869908 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.869886    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpvzq/must-gather-zz546"]
Sep 15 13:39:43.979643 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.979602    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chp5b\" (UniqueName: \"kubernetes.io/projected/1db934fc-0b55-4e1f-a187-1971b0b44efc-kube-api-access-chp5b\") pod \"must-gather-zz546\" (UID: \"1db934fc-0b55-4e1f-a187-1971b0b44efc\") " pod="openshift-must-gather-xpvzq/must-gather-zz546"
Sep 15 13:39:43.979823 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:43.979671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1db934fc-0b55-4e1f-a187-1971b0b44efc-must-gather-output\") pod \"must-gather-zz546\" (UID: \"1db934fc-0b55-4e1f-a187-1971b0b44efc\") " pod="openshift-must-gather-xpvzq/must-gather-zz546"
Sep 15 13:39:44.081248 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.081206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chp5b\" (UniqueName: \"kubernetes.io/projected/1db934fc-0b55-4e1f-a187-1971b0b44efc-kube-api-access-chp5b\") pod \"must-gather-zz546\" (UID: \"1db934fc-0b55-4e1f-a187-1971b0b44efc\") " pod="openshift-must-gather-xpvzq/must-gather-zz546"
Sep 15 13:39:44.081730 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.081258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1db934fc-0b55-4e1f-a187-1971b0b44efc-must-gather-output\") pod \"must-gather-zz546\" (UID: \"1db934fc-0b55-4e1f-a187-1971b0b44efc\") " pod="openshift-must-gather-xpvzq/must-gather-zz546"
Sep 15 13:39:44.081730 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.081598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1db934fc-0b55-4e1f-a187-1971b0b44efc-must-gather-output\") pod \"must-gather-zz546\" (UID: \"1db934fc-0b55-4e1f-a187-1971b0b44efc\") " pod="openshift-must-gather-xpvzq/must-gather-zz546"
Sep 15 13:39:44.095589 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.095565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chp5b\" (UniqueName: \"kubernetes.io/projected/1db934fc-0b55-4e1f-a187-1971b0b44efc-kube-api-access-chp5b\") pod \"must-gather-zz546\" (UID: \"1db934fc-0b55-4e1f-a187-1971b0b44efc\") " pod="openshift-must-gather-xpvzq/must-gather-zz546"
Sep 15 13:39:44.184961 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.184928    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpvzq/must-gather-zz546"
Sep 15 13:39:44.284168 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.284136    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-92knm"]
Sep 15 13:39:44.468227 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.468141    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpvzq/must-gather-zz546"]
Sep 15 13:39:44.471727 ip-10-0-132-169 kubenswrapper[2562]: W0915 13:39:44.471698    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1db934fc_0b55_4e1f_a187_1971b0b44efc.slice/crio-fbe6b7d9762cfff034bb922816a9bf48c54b67b278d89ec30a600124857095ce WatchSource:0}: Error finding container fbe6b7d9762cfff034bb922816a9bf48c54b67b278d89ec30a600124857095ce: Status 404 returned error can't find the container with id fbe6b7d9762cfff034bb922816a9bf48c54b67b278d89ec30a600124857095ce
Sep 15 13:39:44.473499 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.473483    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 13:39:44.640853 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:44.640817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpvzq/must-gather-zz546" event={"ID":"1db934fc-0b55-4e1f-a187-1971b0b44efc","Type":"ContainerStarted","Data":"fbe6b7d9762cfff034bb922816a9bf48c54b67b278d89ec30a600124857095ce"}
Sep 15 13:39:45.649899 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:45.649865    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpvzq/must-gather-zz546" event={"ID":"1db934fc-0b55-4e1f-a187-1971b0b44efc","Type":"ContainerStarted","Data":"5d92618d2648107c8b6e85bf3fbf53d45aa8fdc897f61b931c424db99b03b7fc"}
Sep 15 13:39:45.650329 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:45.649914    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpvzq/must-gather-zz546" event={"ID":"1db934fc-0b55-4e1f-a187-1971b0b44efc","Type":"ContainerStarted","Data":"6d073c011ab1b21037204208f62c2cb96207b96a207f690e5cfd725805ea27d5"}
Sep 15 13:39:45.667862 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:45.667769    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xpvzq/must-gather-zz546" podStartSLOduration=1.7372552909999999 podStartE2EDuration="2.667753649s" totalImagesPullingTime="930.498358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 13:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:39:45.664710582 +0000 UTC m=+3303.457412313" watchObservedRunningTime="2026-09-15 13:39:45.667753649 +0000 UTC m=+3303.460455417"
Sep 15 13:39:47.129294 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:47.129261    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9wmfs_ffa614d2-e094-43ee-a793-042735c1161d/global-pull-secret-syncer/0.log"
Sep 15 13:39:47.306320 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:47.306286    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lbdf9_eedbe00f-27fc-4124-b091-c64793e031d3/konnectivity-agent/0.log"
Sep 15 13:39:47.355525 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:47.355496    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-169.ec2.internal_40993ab6a5e5658e373011e0ad6bc90a/haproxy/0.log"
Sep 15 13:39:52.151573 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:52.151537    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-4grfh_48da0f6b-419d-4bef-841a-25ef4899e8bf/manager/0.log"
Sep 15 13:39:52.272374 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:52.272176    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-9mx7j_d3ca137f-8d7b-4dc1-beac-fd2c4ebb4ae5/manager/0.log"
Sep 15 13:39:52.293246 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:52.293219    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-92knm_2de15519-4f94-4f66-abb2-1fead0097a15/limitador/0.log"
Sep 15 13:39:53.973200 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:53.973163    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-b7j9r_5558222d-f542-4972-b7a1-a6b3425571cd/cluster-monitoring-operator/0.log"
Sep 15 13:39:54.008229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.008192    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-jqcbj_37cdf835-8faf-4180-83e4-40b96a85ca03/kube-state-metrics/0.log"
Sep 15 13:39:54.028873 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.028844    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-jqcbj_37cdf835-8faf-4180-83e4-40b96a85ca03/kube-rbac-proxy-main/0.log"
Sep 15 13:39:54.050086 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.050056    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-jqcbj_37cdf835-8faf-4180-83e4-40b96a85ca03/kube-rbac-proxy-self/0.log"
Sep 15 13:39:54.076945 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.076894    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-784857c55d-j5dnw_a800ca98-8cb1-458d-b95d-0af6ac38e349/metrics-server/0.log"
Sep 15 13:39:54.290142 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.290046    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wzvww_7ff0210f-ee7a-4eb9-bd17-c8396da63af6/node-exporter/0.log"
Sep 15 13:39:54.315993 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.315959    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wzvww_7ff0210f-ee7a-4eb9-bd17-c8396da63af6/kube-rbac-proxy/0.log"
Sep 15 13:39:54.340670 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.340644    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wzvww_7ff0210f-ee7a-4eb9-bd17-c8396da63af6/init-textfile/0.log"
Sep 15 13:39:54.492631 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.492596    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8785bb8a-ab86-40d5-bfd0-f330dcf88b30/prometheus/0.log"
Sep 15 13:39:54.531579 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.531348    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8785bb8a-ab86-40d5-bfd0-f330dcf88b30/config-reloader/0.log"
Sep 15 13:39:54.563130 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.562912    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8785bb8a-ab86-40d5-bfd0-f330dcf88b30/thanos-sidecar/0.log"
Sep 15 13:39:54.597379 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.597308    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8785bb8a-ab86-40d5-bfd0-f330dcf88b30/kube-rbac-proxy-web/0.log"
Sep 15 13:39:54.627228 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.627197    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8785bb8a-ab86-40d5-bfd0-f330dcf88b30/kube-rbac-proxy/0.log"
Sep 15 13:39:54.654679 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.654645    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8785bb8a-ab86-40d5-bfd0-f330dcf88b30/kube-rbac-proxy-thanos/0.log"
Sep 15 13:39:54.677080 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.677054    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8785bb8a-ab86-40d5-bfd0-f330dcf88b30/init-config-reloader/0.log"
Sep 15 13:39:54.708992 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.708956    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-p76dz_a18ffd30-7494-4446-8266-df7dfb5f9365/prometheus-operator/0.log"
Sep 15 13:39:54.733174 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.733138    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-p76dz_a18ffd30-7494-4446-8266-df7dfb5f9365/kube-rbac-proxy/0.log"
Sep 15 13:39:54.931980 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.931952    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cccf48976-k6cbj_4a425049-8fbb-4c61-9017-b9e410487d0a/thanos-query/0.log"
Sep 15 13:39:54.956822 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.956783    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cccf48976-k6cbj_4a425049-8fbb-4c61-9017-b9e410487d0a/kube-rbac-proxy-web/0.log"
Sep 15 13:39:54.978465 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:54.978438    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cccf48976-k6cbj_4a425049-8fbb-4c61-9017-b9e410487d0a/kube-rbac-proxy/0.log"
Sep 15 13:39:55.000672 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:55.000638    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cccf48976-k6cbj_4a425049-8fbb-4c61-9017-b9e410487d0a/prom-label-proxy/0.log"
Sep 15 13:39:55.023313 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:55.023286    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cccf48976-k6cbj_4a425049-8fbb-4c61-9017-b9e410487d0a/kube-rbac-proxy-rules/0.log"
Sep 15 13:39:55.045569 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:55.045516    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cccf48976-k6cbj_4a425049-8fbb-4c61-9017-b9e410487d0a/kube-rbac-proxy-metrics/0.log"
Sep 15 13:39:57.055103 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.055015    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/1.log"
Sep 15 13:39:57.061271 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.061243    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-frn5n_8d0130b6-3e38-41e4-a4dd-00d170040652/console-operator/2.log"
Sep 15 13:39:57.147165 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.147129    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"]
Sep 15 13:39:57.155008 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.154977    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.157806 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.157674    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xpvzq\"/\"default-dockercfg-4dw7n\""
Sep 15 13:39:57.160070 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.159723    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"]
Sep 15 13:39:57.257242 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.257206    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-podres\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.257546 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.257523    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cckzk\" (UniqueName: \"kubernetes.io/projected/a9b21d00-df10-46cf-86ff-75265548ac3c-kube-api-access-cckzk\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.257720 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.257705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-lib-modules\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.257882 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.257868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-sys\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.257988 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.257975    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-proc\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.358728 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358634    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-podres\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.358728 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cckzk\" (UniqueName: \"kubernetes.io/projected/a9b21d00-df10-46cf-86ff-75265548ac3c-kube-api-access-cckzk\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.358960 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358745    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-lib-modules\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.358960 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-sys\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.358960 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-podres\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.358960 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358860    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-sys\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.358960 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358947    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-lib-modules\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.359256 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358951    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-proc\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.359256 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.358994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9b21d00-df10-46cf-86ff-75265548ac3c-proc\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.367784 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.367746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cckzk\" (UniqueName: \"kubernetes.io/projected/a9b21d00-df10-46cf-86ff-75265548ac3c-kube-api-access-cckzk\") pod \"perf-node-gather-daemonset-ljvrs\" (UID: \"a9b21d00-df10-46cf-86ff-75265548ac3c\") " pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.469958 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.469917    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:57.779650 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:57.779528    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"]
Sep 15 13:39:58.290597 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:58.290556    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-cfdkm_4656aefa-4af5-49e1-bd3c-20594a1b1a88/volume-data-source-validator/0.log"
Sep 15 13:39:58.762005 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:58.761968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs" event={"ID":"a9b21d00-df10-46cf-86ff-75265548ac3c","Type":"ContainerStarted","Data":"e9588e81ebc13ceb5b82adf987f71212695517af8dd3e9c19a44ce1286084455"}
Sep 15 13:39:58.762299 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:58.762275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs" event={"ID":"a9b21d00-df10-46cf-86ff-75265548ac3c","Type":"ContainerStarted","Data":"7cc0df07734951d60e1bf5620c9cc6458a9aa88f2713699347c5fff4b8d9f8aa"}
Sep 15 13:39:58.762475 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:58.762424    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:39:58.777127 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:58.777080    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs" podStartSLOduration=1.777022122 podStartE2EDuration="1.777022122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 13:39:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 13:39:58.776389867 +0000 UTC m=+3316.569091600" watchObservedRunningTime="2026-09-15 13:39:58.777022122 +0000 UTC m=+3316.569723855"
Sep 15 13:39:59.440384 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:59.440353    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wnhvg_436023f5-a121-4823-bf16-203409532181/dns/0.log"
Sep 15 13:39:59.457925 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:59.457900    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wnhvg_436023f5-a121-4823-bf16-203409532181/kube-rbac-proxy/0.log"
Sep 15 13:39:59.503239 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:39:59.503210    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nb6bf_3a544b24-4e29-4395-84fe-3d2e1086f6bd/dns-node-resolver/0.log"
Sep 15 13:40:00.110405 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:00.110372    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vq2mc_c3351e98-4182-4ebd-bccd-594fbb589096/node-ca/0.log"
Sep 15 13:40:01.006271 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:01.006228    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78_7a531882-df52-4750-a943-2153d5ae86f9/istio-proxy/0.log"
Sep 15 13:40:01.401327 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:01.401233    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8jxq4_d584efee-f195-4b32-90a9-a158bb292b71/istio-proxy/0.log"
Sep 15 13:40:01.403504 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:01.403476    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8jxq4_d584efee-f195-4b32-90a9-a158bb292b71/istio-proxy/1.log"
Sep 15 13:40:01.657112 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:01.657012    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-89fbc77c4-kzvnf_7ee18f00-83f0-4177-906f-f66a46579c82/router/0.log"
Sep 15 13:40:02.581285 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:02.581250    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bhwk9_95150dd3-0bc2-48d5-b06f-d38ca3f9705e/serve-healthcheck-canary/0.log"
Sep 15 13:40:03.120395 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:03.120366    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-qh8lp_518a335e-be83-4292-82e7-fd571fc39ad1/insights-operator/0.log"
Sep 15 13:40:03.121189 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:03.121168    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-qh8lp_518a335e-be83-4292-82e7-fd571fc39ad1/insights-operator/1.log"
Sep 15 13:40:03.200725 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:03.200699    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jx7t_9efbb847-1884-4528-8390-64cb81e1f483/kube-rbac-proxy/0.log"
Sep 15 13:40:03.218833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:03.218807    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jx7t_9efbb847-1884-4528-8390-64cb81e1f483/exporter/0.log"
Sep 15 13:40:03.237930 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:03.237904    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jx7t_9efbb847-1884-4528-8390-64cb81e1f483/extractor/0.log"
Sep 15 13:40:04.779214 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:04.779183    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xpvzq/perf-node-gather-daemonset-ljvrs"
Sep 15 13:40:05.481942 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:05.481908    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gx4hh_91586217-bf10-4574-a510-9787484c3360/manager/0.log"
Sep 15 13:40:05.697246 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:05.697217    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-flzms_98cfcc4e-4c7d-48dd-90f5-d6d763ea7d1e/manager/0.log"
Sep 15 13:40:05.724165 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:05.724136    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f4dcdf76b-6t6qw_81a93706-a3b7-435c-a6a0-7ebb65663aa9/manager/2.log"
Sep 15 13:40:05.729833 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:05.729806    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f4dcdf76b-6t6qw_81a93706-a3b7-435c-a6a0-7ebb65663aa9/manager/1.log"
Sep 15 13:40:05.842472 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:05.842386    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2pns4_51f724a6-cbd4-47a3-86e7-ef5565c01ec3/manager/0.log"
Sep 15 13:40:05.924229 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:05.924189    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79946fddd6-l9vd4_653dfa3e-9221-4267-b320-e23bfe775222/manager/0.log"
Sep 15 13:40:05.938083 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:05.938053    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79946fddd6-l9vd4_653dfa3e-9221-4267-b320-e23bfe775222/copy-manifests/0.log"
Sep 15 13:40:07.455677 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:07.455648    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c5fdc7d8d-6wsdf_f9b5e605-355b-465a-a753-7d16282e513e/manager/0.log"
Sep 15 13:40:07.637534 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:07.637437    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78_7a531882-df52-4750-a943-2153d5ae86f9/istio-proxy/0.log"
Sep 15 13:40:07.990100 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:07.990065    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8jxq4_d584efee-f195-4b32-90a9-a158bb292b71/istio-proxy/0.log"
Sep 15 13:40:07.992012 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:07.991989    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8jxq4_d584efee-f195-4b32-90a9-a158bb292b71/istio-proxy/1.log"
Sep 15 13:40:08.160704 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:08.160676    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-89fbc77c4-kzvnf_7ee18f00-83f0-4177-906f-f66a46579c82/router/0.log"
Sep 15 13:40:09.727458 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:09.727417    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-brt78_7a531882-df52-4750-a943-2153d5ae86f9/istio-proxy/0.log"
Sep 15 13:40:10.124024 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:10.123926    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8jxq4_d584efee-f195-4b32-90a9-a158bb292b71/istio-proxy/0.log"
Sep 15 13:40:10.126131 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:10.126105    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8jxq4_d584efee-f195-4b32-90a9-a158bb292b71/istio-proxy/1.log"
Sep 15 13:40:10.334337 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:10.334311    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-89fbc77c4-kzvnf_7ee18f00-83f0-4177-906f-f66a46579c82/router/0.log"
Sep 15 13:40:13.415392 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:13.415356    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-bz6pm_255a09b1-7274-435a-8ad0-59ba327862a9/kube-storage-version-migrator-operator/1.log"
Sep 15 13:40:13.416634 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:13.416604    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-bz6pm_255a09b1-7274-435a-8ad0-59ba327862a9/kube-storage-version-migrator-operator/0.log"
Sep 15 13:40:14.697171 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:14.697139    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlghj_2344c353-bb2a-496e-b049-844dd8edf31c/kube-multus-additional-cni-plugins/0.log"
Sep 15 13:40:14.716280 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:14.716254    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlghj_2344c353-bb2a-496e-b049-844dd8edf31c/egress-router-binary-copy/0.log"
Sep 15 13:40:14.736360 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:14.736331    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlghj_2344c353-bb2a-496e-b049-844dd8edf31c/cni-plugins/0.log"
Sep 15 13:40:14.755134 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:14.755112    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlghj_2344c353-bb2a-496e-b049-844dd8edf31c/bond-cni-plugin/0.log"
Sep 15 13:40:14.771621 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:14.771589    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlghj_2344c353-bb2a-496e-b049-844dd8edf31c/routeoverride-cni/0.log"
Sep 15 13:40:14.789648 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:14.789630    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlghj_2344c353-bb2a-496e-b049-844dd8edf31c/whereabouts-cni-bincopy/0.log"
Sep 15 13:40:14.807862 ip-10-0-132-169 kubenswrapper[2562]: I0915 13:40:14.807842    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlghj_2344c353-bb2a-496e-b049-844dd8edf31c/whereabouts-cni/0.log"