Sep 11 13:46:40.041979 ip-10-0-137-192 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 13:46:40.041992 ip-10-0-137-192 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 13:46:40.042004 ip-10-0-137-192 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 13:46:40.042331 ip-10-0-137-192 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 13:46:50.290704 ip-10-0-137-192 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 13:46:50.290723 ip-10-0-137-192 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 193398ad96034c6da86923747edfeef3 --
Sep 11 13:48:22.061693 ip-10-0-137-192 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:48:22.519136 ip-10-0-137-192 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:48:22.519136 ip-10-0-137-192 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:48:22.519136 ip-10-0-137-192 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:48:22.519136 ip-10-0-137-192 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:48:22.519136 ip-10-0-137-192 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:48:22.520448 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.520353    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:48:22.524605 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524585    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:22.524605 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524605    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524610    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524613    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524617    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524620    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524623    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524626    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524629    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524632    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524635    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524637    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524640    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524647    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524650    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524653    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524655    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524658    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524661    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524664    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:22.524681 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524666    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524669    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524672    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524675    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524678    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524680    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524683    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524686    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524689    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524692    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524696    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524700    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524705    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524708    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524710    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524713    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524716    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524719    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524721    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:22.525148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524724    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524727    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524729    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524732    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524734    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524737    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524739    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524742    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524745    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524748    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524751    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524753    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524755    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524758    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524762    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524765    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524768    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524770    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524773    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524775    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:22.525633 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524778    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524780    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524783    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524786    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524788    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524791    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524793    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524796    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524798    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524800    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524803    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524806    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524808    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524810    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524813    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524815    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524818    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524821    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524823    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524826    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:22.526129 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524833    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524836    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524839    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524841    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524844    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524846    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524849    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524853    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524856    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524858    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524861    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524864    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524866    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524869    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524875    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524878    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524881    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524885    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524889    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.524892    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:22.526639 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525664    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525673    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525677    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525680    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525683    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525686    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525689    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525692    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525695    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525697    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525700    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525704    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525708    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525714    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525717    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525720    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525723    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525725    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525729    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:22.527166 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525732    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525734    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525737    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525740    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525742    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525745    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525747    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525750    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525753    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525756    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525758    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525761    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525764    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525766    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525769    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525771    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525774    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525777    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525779    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:22.527668 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525782    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525785    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525787    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525790    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525793    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525795    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525798    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525800    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525803    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525805    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525809    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525811    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525814    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525817    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525820    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525822    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525824    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525827    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525830    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525832    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:22.528132 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525835    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525838    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525841    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525844    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525846    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525849    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525851    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525854    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525856    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525859    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525861    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525864    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525867    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525869    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525871    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525874    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525876    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525879    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525881    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525883    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:22.528651 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525886    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525889    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525891    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525896    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525899    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525901    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525903    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525906    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525908    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525911    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525913    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525916    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525918    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525920    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525923    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525925    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525928    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525930    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525933    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525936    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:22.529148 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.525938    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526335    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526340    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526343    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526346    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526349    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526352    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526354    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526357    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526359    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526362    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526364    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526367    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526370    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526373    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526378    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526381    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526383    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526386    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526389    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:22.529742 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526391    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526394    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526398    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526402    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526406    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526409    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526412    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526415    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526418    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526420    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526423    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526425    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526428    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526430    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526433    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526435    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526438    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526440    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526443    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:22.530242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526445    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526448    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526451    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526455    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526457    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526460    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526462    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526465    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526469    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526471    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526474    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526477    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526479    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526482    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526484    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526487    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526489    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526497    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526500    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526502    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:22.530734 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526505    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526507    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526510    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526512    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526515    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526517    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526520    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526523    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526525    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526528    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526530    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526533    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526535    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526537    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526540    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526544    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526547    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526550    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526553    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:22.531218 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526555    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526560    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526562    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526565    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526568    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526571    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526574    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526577    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526579    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526582    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526585    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526588    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526590    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526592    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526595    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526597    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526600    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526602    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526605    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526607    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:22.531695 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526610    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.526612    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526697    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526738    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526744    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526749    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526753    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526756    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526761    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526765    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526768    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526771    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526775    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526779    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526783    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526786    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526789    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526792    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526795    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526798    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526802    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526805    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526809    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526812    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:48:22.532245 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526816    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526819    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526823    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526825    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526829    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526832    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526835    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526838    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526841    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526844    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526848    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526851    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526853    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526856    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526859    2563 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526862    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526866    2563 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526869    2563 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526872    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526875    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526878    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526884    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526887    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526890    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526893    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:48:22.532858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526896    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526900    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526903    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526906    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526908    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526912    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526915    2563 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526919    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526922    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526925    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526928    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526931    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526934    2563 flags.go:64] FLAG: --help="false"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526937    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526940    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526943    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526947    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.526950    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527000    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527004    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527007    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527010    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527014    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:48:22.533527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527017    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527020    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527023    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527025    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527028    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527032    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527035    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527038    2563 flags.go:64] FLAG: --lock-file=""
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527042    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527045    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527048    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527054    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527056    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527059    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527063    2563 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527066    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527070    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527073    2563 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527075    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527080    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527083    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527087    2563 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527090    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527093    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527096    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:48:22.534134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527099    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527102    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527105    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527108    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527115    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527118    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527121    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527124    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527127    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527133    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527136    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527139    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527142    2563 flags.go:64] FLAG: --port="10250"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527147    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527150    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ecdda8d9d11f11d7"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527153    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527156    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527159    2563 flags.go:64] FLAG: --register-node="true"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527162    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527166    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527168    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527174    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527177    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527180    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527183    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:48:22.534779 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527186    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527189    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527192    2563 flags.go:64] FLAG: --runonce="false"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527194    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527197    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527200    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527203    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527206    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527209    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527213    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527216    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527218    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527221    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527224    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527227    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527230    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527233    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527235    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527241    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527243    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527247    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527251    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527254    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527256    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527260    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:48:22.535461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527263    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527266    2563 flags.go:64] FLAG: --v="2"
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527270    2563 flags.go:64] FLAG: --version="false"
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527275    2563 flags.go:64] FLAG: --vmodule=""
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527279    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.527282    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527415    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527419    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527423    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527426    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527428    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527431    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527433    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527436    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527439    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527441    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527444    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527446    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527450    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527453    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527456    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:22.536428 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527458    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527461    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527463    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527466    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527468    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527470    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527473    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527477    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527481    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527484    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527487    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527490    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527493    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527496    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527499    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527502    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527504    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527507    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527509    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:22.537165 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527512    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527514    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527517    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527519    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527521    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527524    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527526    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527529    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527531    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527534    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527537    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527539    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527542    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527544    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527547    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527549    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527551    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527554    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527557    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527561    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:22.537678 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527564    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527567    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527570    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527572    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527575    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527578    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527581    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527584    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527586    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527589    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527592    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527594    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527597    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527600    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527602    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527605    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527607    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527610    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527612    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527615    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:22.538185 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527617    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527620    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527622    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527625    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527627    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527629    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527632    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527634    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527637    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527639    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527642    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527644    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527647    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527649    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527652    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527654    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527657    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527659    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527662    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527666    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:22.538698 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527670    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527674    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527678    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527682    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.527685    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.528358    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.537860    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.537880    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.538880    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.538889    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538950    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538957    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538960    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538963    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:22.539195 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538966    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538971    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538975    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538978    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538981    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538984    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538987    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538990    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538993    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538996    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.538999    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539001    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539004    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539007    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539010    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539013    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539017    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539019    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539022    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:22.539617 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539024    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539027    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539030    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539033    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539035    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539038    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539041    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539044    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539047    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539049    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539052    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539054    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539057    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539059    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539064    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539067    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539070    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539074    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539078    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:22.540076 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539082    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539085    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539088    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539090    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539093    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539096    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539098    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539101    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539104    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539106    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539110    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539112    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539115    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539117    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539120    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539123    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539125    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539128    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539130    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539133    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:22.540569 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539136    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539139    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539142    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539144    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539147    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539149    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539152    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539155    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539157    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539160    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539162    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539165    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539167    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539170    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539172    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539174    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539177    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539180    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539182    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539185    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:22.541072 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539187    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539190    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539192    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539195    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539198    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539200    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539203    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539206    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539208    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539211    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539214    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539216    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539219    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539221    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539224    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539226    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:22.541650 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539229    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.539234    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539370    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539376    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539379    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539381    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539384    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539387    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539390    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539393    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539396    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539399    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539402    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539404    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539407    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:22.542071 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539410    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539413    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539415    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539418    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539420    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539423    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539425    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539428    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539430    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539433    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539435    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539438    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539441    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539444    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539446    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539449    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539452    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539454    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539457    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539459    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:22.542465 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539462    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539464    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539492    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539496    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539499    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539502    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539505    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539508    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539511    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539514    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539516    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539519    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539524    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539528    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539531    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539534    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539537    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539539    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539542    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:22.542953 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539545    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539547    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539550    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539552    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539555    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539557    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539560    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539563    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539565    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539568    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539570    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539573    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539575    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539578    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539581    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539584    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539586    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539589    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539593    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:22.543416 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539596    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539599    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539602    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539604    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539607    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539609    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539612    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539615    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539617    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539620    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539622    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539625    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539628    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539631    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539633    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539636    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539639    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539641    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539644    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539648    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:22.543882 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539651    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539653    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539655    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539658    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539661    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539663    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539666    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:22.539668    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.539673    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.539800    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:48:22.544402 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.544304    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:48:22.545434 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.545419    2563 server.go:1019] "Starting client certificate rotation"
Sep 11 13:48:22.545547 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.545527    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:48:22.545576 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.545564    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:48:22.548372 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.548356    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:48:22.549609 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.549588    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:48:22.561509 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.561487    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:48:22.564958 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.564942    2563 log.go:25] "Validated CRI v1 image API"
Sep 11 13:48:22.565919 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.565901    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:48:22.570411 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.570387    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8493c92d-f35a-47fa-b934-3cf4947680a8:/dev/nvme0n1p4 e2bb205f-d0f3-4385-a30e-f16c184d8a03:/dev/nvme0n1p3]
Sep 11 13:48:22.570479 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.570411    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:48:22.577605 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.577234    2563 manager.go:216] Machine: {Timestamp:2026-09-11 13:48:22.576246488 +0000 UTC m=+0.383879323 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3067185 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec294f3536c4c879063cb533434cf9ff SystemUUID:ec294f35-36c4-c879-063c-b533434cf9ff BootID:193398ad-9603-4c6d-a869-23747edfeef3 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d6:00:6b:03:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d6:00:6b:03:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:96:f6:9d:c6:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:48:22.577605 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.577578    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:48:22.577793 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.577743    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:48:22.579144 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.579124    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:48:22.581354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.581305    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:48:22.581504 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.581360    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-192.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:48:22.582260 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.582249    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:48:22.582297 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.582262    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:48:22.582297 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.582270    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:48:22.582297 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.582286    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:48:22.583412 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.583401    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:48:22.586900 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.586884    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:48:22.587035 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.587026    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:48:22.587076 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.587058    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:48:22.587109 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.587084    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:48:22.587109 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.587097    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:48:22.587193 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.587110    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:48:22.587193 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.587123    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:48:22.587635 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.587620    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.587779    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.590522    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.593660    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.593689    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595756    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595770    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595776    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595784    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595797    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595806    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595820    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595825    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595834    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595840    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595849    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.595854    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.596692    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:48:22.598954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.596701    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:48:22.601265 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.601249    2563 server.go:1268] "Started kubelet"
Sep 11 13:48:22.601370 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.601333    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:48:22.601467 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.601407    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:48:22.601501 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.601493    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:48:22.602300 ip-10-0-137-192 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:48:22.603014 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.602992    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:48:22.604255 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.604241    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:48:22.605332 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.605295    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-192.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:48:22.605432 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.605383    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:48:22.605491 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.605453    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:48:22.606270 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.606251    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:48:22.607982 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.606879    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-192.ec2.internal.18d44859d2f0c714  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-192.ec2.internal,UID:ip-10-0-137-192.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-192.ec2.internal,},FirstTimestamp:2026-09-11 13:48:22.601213716 +0000 UTC m=+0.408846552,LastTimestamp:2026-09-11 13:48:22.601213716 +0000 UTC m=+0.408846552,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-192.ec2.internal,}"
Sep 11 13:48:22.611091 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.611069    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:48:22.611091 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.611081    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:48:22.611241 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.611135    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:22.611866 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.611847    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:48:22.611950 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.611872    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:48:22.611950 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.611850    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:48:22.612054 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612008    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:48:22.612054 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612017    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:48:22.612631 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612610    2563 factory.go:55] Registering systemd factory
Sep 11 13:48:22.612719 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612660    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:48:22.612896 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612882    2563 factory.go:153] Registering CRI-O factory
Sep 11 13:48:22.612929 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612900    2563 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:48:22.612978 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612955    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:48:22.612978 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612975    2563 factory.go:103] Registering Raw factory
Sep 11 13:48:22.613077 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.612998    2563 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:48:22.613428 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.613411    2563 manager.go:349] Starting recovery of all containers
Sep 11 13:48:22.616450 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.616145    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:48:22.616450 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.616237    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:48:22.625269 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.625231    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gqcdb"
Sep 11 13:48:22.625426 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.625292    2563 manager.go:354] Recovery completed
Sep 11 13:48:22.626825 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.626758    2563 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 11 13:48:22.629911 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.629896    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:22.631954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.631939    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:22.631997 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.631971    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:22.631997 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.631983    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:22.632595 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.632582    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:48:22.632595 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.632595    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:48:22.632691 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.632615    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:48:22.632691 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.632645    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:22.632766 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.632750    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gqcdb"
Sep 11 13:48:22.633361 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.633348    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:22.633413 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.633376    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:22.633413 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.633387    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:22.634159 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.634061    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-192.ec2.internal.18d44859d4c5dc62  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-192.ec2.internal,UID:ip-10-0-137-192.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-192.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-192.ec2.internal,},FirstTimestamp:2026-09-11 13:48:22.631955554 +0000 UTC m=+0.439588389,LastTimestamp:2026-09-11 13:48:22.631955554 +0000 UTC m=+0.439588389,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-192.ec2.internal,}"
Sep 11 13:48:22.634238 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.634225    2563 policy_none.go:49] "None policy: Start"
Sep 11 13:48:22.634281 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.634247    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:48:22.634281 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.634266    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:48:22.635502 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.635487    2563 policy_none.go:47] "Start"
Sep 11 13:48:22.682952 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.682921    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.684213    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.684271    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.684299    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.684361    2563 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.686786    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.687140    2563 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.687176    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.687188    2563 server.go:87] "Starting device plugin registration server"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.687696    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.687742    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.687874    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.687975    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.687987    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.688642    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:48:22.691542 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.688695    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:22.784936 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.784807    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal"]
Sep 11 13:48:22.784936 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.784923    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:22.786821 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.786802    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:22.786926 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.786830    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:22.786926 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.786841    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:22.787946 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.787933    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:22.788167 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.788154    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:22.788267 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.788250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.788304 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.788281    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:22.789051 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789027    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:22.789117 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789063    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:22.789117 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789083    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:22.789188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789124    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.789188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789129    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:22.789188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789144    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:22.789188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789156    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:22.789188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789161    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:22.789188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789183    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:22.789435 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.789267    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:22.790499 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.790486    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.790540 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.790513    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:22.791932 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.791913    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:22.792064 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.791955    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:22.792064 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.791977    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:22.797302 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.797287    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.797302 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.797309    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-192.ec2.internal\": node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:22.815218 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.815190    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-192.ec2.internal\" not found" node="ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.820043 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.820017    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-192.ec2.internal\" not found" node="ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.822588 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.822571    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:22.913832 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.913786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f66f75ce0b3c443239a1b57d427def08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal\" (UID: \"f66f75ce0b3c443239a1b57d427def08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.913832 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.913827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f66f75ce0b3c443239a1b57d427def08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal\" (UID: \"f66f75ce0b3c443239a1b57d427def08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.913832 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:22.913845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f63e32dd172790f2b9a6ee29b1c6f139-config\") pod \"kube-apiserver-proxy-ip-10-0-137-192.ec2.internal\" (UID: \"f63e32dd172790f2b9a6ee29b1c6f139\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:22.922937 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:22.922906    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:23.014371 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.014307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f63e32dd172790f2b9a6ee29b1c6f139-config\") pod \"kube-apiserver-proxy-ip-10-0-137-192.ec2.internal\" (UID: \"f63e32dd172790f2b9a6ee29b1c6f139\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.014371 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.014372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f66f75ce0b3c443239a1b57d427def08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal\" (UID: \"f66f75ce0b3c443239a1b57d427def08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.014495 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.014388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f66f75ce0b3c443239a1b57d427def08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal\" (UID: \"f66f75ce0b3c443239a1b57d427def08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.014495 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.014434    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f66f75ce0b3c443239a1b57d427def08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal\" (UID: \"f66f75ce0b3c443239a1b57d427def08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.014495 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.014444    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f63e32dd172790f2b9a6ee29b1c6f139-config\") pod \"kube-apiserver-proxy-ip-10-0-137-192.ec2.internal\" (UID: \"f63e32dd172790f2b9a6ee29b1c6f139\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.014495 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.014480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f66f75ce0b3c443239a1b57d427def08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal\" (UID: \"f66f75ce0b3c443239a1b57d427def08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.023446 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:23.023420    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:23.119353 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.119263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.123556 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:23.123528    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:23.123681 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.123558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.224229 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:23.224191    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:23.324735 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:23.324705    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:23.425461 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:23.425375    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:23.526017 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:23.525985    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:23.545145 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.545115    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:48:23.545314 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.545284    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 13:48:23.626252 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:23.626207    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-192.ec2.internal\" not found"
Sep 11 13:48:23.627890 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:23.627858    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66f75ce0b3c443239a1b57d427def08.slice/crio-bbe9e93f9fa01e2f162310d4481c2e8c326df284b1d80e5c1dd5f7db544b5711 WatchSource:0}: Error finding container bbe9e93f9fa01e2f162310d4481c2e8c326df284b1d80e5c1dd5f7db544b5711: Status 404 returned error can't find the container with id bbe9e93f9fa01e2f162310d4481c2e8c326df284b1d80e5c1dd5f7db544b5711
Sep 11 13:48:23.633917 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.633893    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:48:23.636527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.636503    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 13:43:24 +0000 UTC" deadline="2028-04-18 19:06:11.154204722 +0000 UTC"
Sep 11 13:48:23.636577 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.636528    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14045h17m47.517679832s"
Sep 11 13:48:23.688599 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.688484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal" event={"ID":"f63e32dd172790f2b9a6ee29b1c6f139","Type":"ContainerStarted","Data":"1d80ea0325ad0bda191ce103d1aef6d1784a1623170ca849a6da908bb5ca1826"}
Sep 11 13:48:23.689533 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.689508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal" event={"ID":"f66f75ce0b3c443239a1b57d427def08","Type":"ContainerStarted","Data":"bbe9e93f9fa01e2f162310d4481c2e8c326df284b1d80e5c1dd5f7db544b5711"}
Sep 11 13:48:23.722290 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.722262    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:23.811233 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.811200    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.821660 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.821640    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:48:23.823757 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.823742    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal"
Sep 11 13:48:23.885683 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.885658    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:23.908786 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:23.908766    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:48:25.598858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:25.598773    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:26.137178 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.137131    2563 apiserver.go:52] "Watching apiserver"
Sep 11 13:48:26.147679 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.147643    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:48:26.148121 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.148083    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-trgt2","openshift-cluster-node-tuning-operator/tuned-j6dmp","openshift-dns/node-resolver-2tdnj","openshift-multus/multus-additional-cni-plugins-wdlff","openshift-network-diagnostics/network-check-target-sfs7q","openshift-network-operator/iptables-alerter-454wg","openshift-ovn-kubernetes/ovnkube-node-4xs2f","kube-system/global-pull-secret-syncer-dg5cq","kube-system/konnectivity-agent-x4cs8","kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh","openshift-image-registry/node-ca-nns9f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal","openshift-multus/multus-4s6pt"]
Sep 11 13:48:26.149925 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.149900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.151290 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.151269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.152905 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.152885    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:48:26.153097 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.153039    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:48:26.153178 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.153148    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:48:26.153297 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.153275    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:48:26.153389 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.153296    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nrtzh\""
Sep 11 13:48:26.153389 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.153350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.153701 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.153673    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:48:26.153824 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.153806    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:48:26.153969 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.153951    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:48:26.156772 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.154882    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:48:26.156772 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.155045    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:48:26.156772 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.155402    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-78m7q\""
Sep 11 13:48:26.156772 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.155554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.157066 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.157048    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:48:26.157368 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.157352    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:48:26.157607 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.157592    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hfrnc\""
Sep 11 13:48:26.158248 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.158051    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9lplj\""
Sep 11 13:48:26.158395 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.158375    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:48:26.158473 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.158440    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:26.158560 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.158531    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:26.158928 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.158911    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:48:26.159744 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.159723    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:48:26.159854 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.159836    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:48:26.159957 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.159728    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:48:26.160131 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.160111    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.161475 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.161454    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:48:26.161656 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.161641    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:26.162451 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.162428    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:48:26.162523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.162452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x4c9z\""
Sep 11 13:48:26.162775 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.162740    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:48:26.162903 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.162862    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:48:26.163243 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.163226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.164069 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.164050    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:48:26.164291 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.164275    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:48:26.164356 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.164276    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-schgd\""
Sep 11 13:48:26.164529 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.164514    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.165525 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.165505    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:48:26.166010 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.165991    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.166101 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.166056    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:26.166306 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.166292    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hrtrm\""
Sep 11 13:48:26.166876 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.166858    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:48:26.167195 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.167176    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:48:26.167413 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.167391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:26.167473 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.167454    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:26.167626 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.167612    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ppkxx\""
Sep 11 13:48:26.168974 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.168957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.171358 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.171335    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:48:26.171436 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.171402    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:48:26.171507 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.171335    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:48:26.171598 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.171577    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-df6pg\""
Sep 11 13:48:26.173447 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173429    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovn-node-metrics-cert\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.173533 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78099fbe-6524-4477-8da7-1184017e0ed8-tmp-dir\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.173533 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-host-slash\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.173533 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-registration-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.173533 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-var-lib-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.173734 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-node-log\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.173734 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.173734 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovnkube-script-lib\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.173734 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173669    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78099fbe-6524-4477-8da7-1184017e0ed8-hosts-file\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.173734 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.173998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:26.173998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-run-netns\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.173998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.173998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr5gs\" (UniqueName: \"kubernetes.io/projected/78099fbe-6524-4477-8da7-1184017e0ed8-kube-api-access-dr5gs\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.173998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-slash\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.173998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf2sn\" (UniqueName: \"kubernetes.io/projected/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-kube-api-access-jf2sn\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.173998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-system-cni-dir\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.173998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.173981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-iptables-alerter-script\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-ovn\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovnkube-config\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-cnibin\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174106    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db06779b-7d1e-4c26-b85c-1346bb5a1a01-konnectivity-ca\") pod \"konnectivity-agent-x4cs8\" (UID: \"db06779b-7d1e-4c26-b85c-1346bb5a1a01\") " pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-log-socket\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-env-overrides\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-etc-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6crr\" (UniqueName: \"kubernetes.io/projected/75860117-a52f-47e3-8ab9-bc7d71df6bb1-kube-api-access-c6crr\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174266    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gsh\" (UniqueName: \"kubernetes.io/projected/12584014-59b3-4e2c-97e7-56c654c6bca6-kube-api-access-p2gsh\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-socket-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.174352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2pwb\" (UniqueName: \"kubernetes.io/projected/547be90d-fcb6-4db0-ae80-181de31c87b5-kube-api-access-m2pwb\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-kubelet\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-cni-bin\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-cni-netd\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-device-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-systemd\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db06779b-7d1e-4c26-b85c-1346bb5a1a01-agent-certs\") pod \"konnectivity-agent-x4cs8\" (UID: \"db06779b-7d1e-4c26-b85c-1346bb5a1a01\") " pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-sys-fs\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-systemd-units\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.174841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.174671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.175592 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.175571    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:48:26.196228 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.196202    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cxd2g"
Sep 11 13:48:26.204365 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.204295    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cxd2g"
Sep 11 13:48:26.262652 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.262626    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:48:26.275300 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.275300 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.275515 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-sys\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.275515 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-cni-binary-copy\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.275515 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-cni-bin\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.275515 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-cni-multus\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.275515 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-systemd-units\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275688 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovn-node-metrics-cert\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275688 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78099fbe-6524-4477-8da7-1184017e0ed8-tmp-dir\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.275688 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-systemd-units\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275688 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-registration-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.275688 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-var-lib-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275688 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275688 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovnkube-script-lib\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-modprobe-d\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-registration-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-var-lib-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-system-cni-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275805    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-kubelet\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78099fbe-6524-4477-8da7-1184017e0ed8-tmp-dir\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-kubernetes\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.275935 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-system-cni-dir\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovnkube-config\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-cnibin\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.275976    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-etc-kubernetes\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-system-cni-dir\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-log-socket\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-var-lib-kubelet\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-log-socket\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-netns\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jntxf\" (UniqueName: \"kubernetes.io/projected/df547ee7-ed80-4d81-b8e0-b46339dae222-kube-api-access-jntxf\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2adefd72-0d9f-40aa-a193-6b68449ce681-host\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-etc-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovnkube-script-lib\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysctl-conf\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276286    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-lib-modules\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.276309 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276293    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-etc-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-cni-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbrht\" (UniqueName: \"kubernetes.io/projected/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-kube-api-access-rbrht\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2pwb\" (UniqueName: \"kubernetes.io/projected/547be90d-fcb6-4db0-ae80-181de31c87b5-kube-api-access-m2pwb\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5fa3726c-9b76-4680-8d8b-0931a221c76f-dbus\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276403    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-hostroot\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovnkube-config\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-systemd\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-systemd\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db06779b-7d1e-4c26-b85c-1346bb5a1a01-agent-certs\") pod \"konnectivity-agent-x4cs8\" (UID: \"db06779b-7d1e-4c26-b85c-1346bb5a1a01\") " pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-host\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276682    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-socket-dir-parent\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-sys-fs\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.277023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-host-slash\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-sys-fs\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-host-slash\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-openvswitch\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-node-log\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78099fbe-6524-4477-8da7-1184017e0ed8-hosts-file\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276961    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-node-log\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.276982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efecc3fc-ad7a-4159-9c19-0d395f4c074c-tmp\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78099fbe-6524-4477-8da7-1184017e0ed8-hosts-file\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-conf-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-run-netns\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr5gs\" (UniqueName: \"kubernetes.io/projected/78099fbe-6524-4477-8da7-1184017e0ed8-kube-api-access-dr5gs\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.278160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysconfig\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-run-netns\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-run\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-multus-certs\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-slash\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf2sn\" (UniqueName: \"kubernetes.io/projected/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-kube-api-access-jf2sn\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-iptables-alerter-script\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-slash\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-ovn\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5fa3726c-9b76-4680-8d8b-0931a221c76f-kubelet-config\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-run-ovn\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2adefd72-0d9f-40aa-a193-6b68449ce681-serviceca\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-cnibin\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12584014-59b3-4e2c-97e7-56c654c6bca6-cnibin\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db06779b-7d1e-4c26-b85c-1346bb5a1a01-konnectivity-ca\") pod \"konnectivity-agent-x4cs8\" (UID: \"db06779b-7d1e-4c26-b85c-1346bb5a1a01\") " pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:26.278855 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-env-overrides\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-tuned\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277563    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b72zd\" (UniqueName: \"kubernetes.io/projected/efecc3fc-ad7a-4159-9c19-0d395f4c074c-kube-api-access-b72zd\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6crr\" (UniqueName: \"kubernetes.io/projected/75860117-a52f-47e3-8ab9-bc7d71df6bb1-kube-api-access-c6crr\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysctl-d\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-systemd\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-daemon-config\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kpnq\" (UniqueName: \"kubernetes.io/projected/2adefd72-0d9f-40aa-a193-6b68449ce681-kube-api-access-7kpnq\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gsh\" (UniqueName: \"kubernetes.io/projected/12584014-59b3-4e2c-97e7-56c654c6bca6-kube-api-access-p2gsh\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-socket-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-kubelet\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-cni-bin\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-kubelet\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-cni-netd\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-k8s-cni-cncf-io\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-device-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.279538 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.277957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-iptables-alerter-script\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.280308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.278007    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-cni-netd\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.280308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.278029    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75860117-a52f-47e3-8ab9-bc7d71df6bb1-env-overrides\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.280308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.278057    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-device-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.280308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.278080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75860117-a52f-47e3-8ab9-bc7d71df6bb1-host-cni-bin\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.280308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.278105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/547be90d-fcb6-4db0-ae80-181de31c87b5-socket-dir\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.280308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.278111    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12584014-59b3-4e2c-97e7-56c654c6bca6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.280308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.278123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db06779b-7d1e-4c26-b85c-1346bb5a1a01-konnectivity-ca\") pod \"konnectivity-agent-x4cs8\" (UID: \"db06779b-7d1e-4c26-b85c-1346bb5a1a01\") " pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:26.281405 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.281384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75860117-a52f-47e3-8ab9-bc7d71df6bb1-ovn-node-metrics-cert\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.281499 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.281481    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db06779b-7d1e-4c26-b85c-1346bb5a1a01-agent-certs\") pod \"konnectivity-agent-x4cs8\" (UID: \"db06779b-7d1e-4c26-b85c-1346bb5a1a01\") " pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:26.282870 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.282845    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:26.283410 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.283392    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:26.283499 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.283415    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j72c5 for pod openshift-network-diagnostics/network-check-target-sfs7q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:26.283543 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.283505    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5 podName:5968c2e1-979b-4278-8e7a-dd70729bf698 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:26.783484077 +0000 UTC m=+3.041684145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j72c5" (UniqueName: "kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5") pod "network-check-target-sfs7q" (UID: "5968c2e1-979b-4278-8e7a-dd70729bf698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:26.285072 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.285044    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr5gs\" (UniqueName: \"kubernetes.io/projected/78099fbe-6524-4477-8da7-1184017e0ed8-kube-api-access-dr5gs\") pod \"node-resolver-2tdnj\" (UID: \"78099fbe-6524-4477-8da7-1184017e0ed8\") " pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.285653 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.285624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2pwb\" (UniqueName: \"kubernetes.io/projected/547be90d-fcb6-4db0-ae80-181de31c87b5-kube-api-access-m2pwb\") pod \"aws-ebs-csi-driver-node-xmlkh\" (UID: \"547be90d-fcb6-4db0-ae80-181de31c87b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.285958 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.285942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf2sn\" (UniqueName: \"kubernetes.io/projected/6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77-kube-api-access-jf2sn\") pod \"iptables-alerter-454wg\" (UID: \"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77\") " pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.286225 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.286200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gsh\" (UniqueName: \"kubernetes.io/projected/12584014-59b3-4e2c-97e7-56c654c6bca6-kube-api-access-p2gsh\") pod \"multus-additional-cni-plugins-wdlff\" (UID: \"12584014-59b3-4e2c-97e7-56c654c6bca6\") " pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.287413 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.287392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6crr\" (UniqueName: \"kubernetes.io/projected/75860117-a52f-47e3-8ab9-bc7d71df6bb1-kube-api-access-c6crr\") pod \"ovnkube-node-4xs2f\" (UID: \"75860117-a52f-47e3-8ab9-bc7d71df6bb1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.378998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.378956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efecc3fc-ad7a-4159-9c19-0d395f4c074c-tmp\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.378998 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-conf-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysconfig\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-run\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-multus-certs\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.379163    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-conf-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysconfig\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5fa3726c-9b76-4680-8d8b-0931a221c76f-kubelet-config\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379221    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5fa3726c-9b76-4680-8d8b-0931a221c76f-kubelet-config\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.379251 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.379246    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret podName:5fa3726c-9b76-4680-8d8b-0931a221c76f nodeName:}" failed. No retries permitted until 2026-09-11 13:48:26.879227239 +0000 UTC m=+3.137427249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret") pod "global-pull-secret-syncer-dg5cq" (UID: "5fa3726c-9b76-4680-8d8b-0931a221c76f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379267    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-run\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-multus-certs\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2adefd72-0d9f-40aa-a193-6b68449ce681-serviceca\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-tuned\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b72zd\" (UniqueName: \"kubernetes.io/projected/efecc3fc-ad7a-4159-9c19-0d395f4c074c-kube-api-access-b72zd\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysctl-d\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.379373    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-systemd\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-daemon-config\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.379437    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs podName:df547ee7-ed80-4d81-b8e0-b46339dae222 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:26.879419564 +0000 UTC m=+3.137619571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs") pod "network-metrics-daemon-trgt2" (UID: "df547ee7-ed80-4d81-b8e0-b46339dae222") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kpnq\" (UniqueName: \"kubernetes.io/projected/2adefd72-0d9f-40aa-a193-6b68449ce681-kube-api-access-7kpnq\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-systemd\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-k8s-cni-cncf-io\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-sys\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-cni-binary-copy\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.379816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-cni-bin\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-cni-multus\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-modprobe-d\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-system-cni-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-kubelet\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-kubernetes\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-cnibin\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-etc-kubernetes\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-var-lib-kubelet\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-netns\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntxf\" (UniqueName: \"kubernetes.io/projected/df547ee7-ed80-4d81-b8e0-b46339dae222-kube-api-access-jntxf\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2adefd72-0d9f-40aa-a193-6b68449ce681-serviceca\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2adefd72-0d9f-40aa-a193-6b68449ce681-host\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysctl-d\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-daemon-config\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysctl-conf\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379736    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-sys\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-lib-modules\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.380523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.379697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-k8s-cni-cncf-io\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-cni-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrht\" (UniqueName: \"kubernetes.io/projected/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-kube-api-access-rbrht\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-cni-bin\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-var-lib-kubelet\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5fa3726c-9b76-4680-8d8b-0931a221c76f-dbus\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380255    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-kubernetes\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380262    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-kubelet\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-hostroot\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-host\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-cnibin\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-socket-dir-parent\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-lib-modules\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-etc-kubernetes\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380365    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-run-netns\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380386    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2adefd72-0d9f-40aa-a193-6b68449ce681-host\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-cni-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5fa3726c-9b76-4680-8d8b-0931a221c76f-dbus\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.381352 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380485    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-host\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.382081 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-sysctl-conf\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.382081 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-system-cni-dir\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.382081 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380529    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-multus-socket-dir-parent\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.382081 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-hostroot\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.382081 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380548    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-host-var-lib-cni-multus\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.382081 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-cni-binary-copy\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.382081 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.380592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-modprobe-d\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.382385 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.382178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efecc3fc-ad7a-4159-9c19-0d395f4c074c-etc-tuned\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.382385 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.382200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efecc3fc-ad7a-4159-9c19-0d395f4c074c-tmp\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.391020 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.390924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b72zd\" (UniqueName: \"kubernetes.io/projected/efecc3fc-ad7a-4159-9c19-0d395f4c074c-kube-api-access-b72zd\") pod \"tuned-j6dmp\" (UID: \"efecc3fc-ad7a-4159-9c19-0d395f4c074c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.392041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.392012    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbrht\" (UniqueName: \"kubernetes.io/projected/6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b-kube-api-access-rbrht\") pod \"multus-4s6pt\" (UID: \"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b\") " pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.393255 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.393219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jntxf\" (UniqueName: \"kubernetes.io/projected/df547ee7-ed80-4d81-b8e0-b46339dae222-kube-api-access-jntxf\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:26.394977 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.394955    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kpnq\" (UniqueName: \"kubernetes.io/projected/2adefd72-0d9f-40aa-a193-6b68449ce681-kube-api-access-7kpnq\") pod \"node-ca-nns9f\" (UID: \"2adefd72-0d9f-40aa-a193-6b68449ce681\") " pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.468278 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.468237    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:26.476279 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.476248    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh"
Sep 11 13:48:26.484306 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.484280    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2tdnj"
Sep 11 13:48:26.491031 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.491004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wdlff"
Sep 11 13:48:26.500867 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.500843    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-454wg"
Sep 11 13:48:26.508627 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.508606    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:26.516429 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.516411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4s6pt"
Sep 11 13:48:26.525290 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.525263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j6dmp"
Sep 11 13:48:26.531246 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.531227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nns9f"
Sep 11 13:48:26.641087 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.640999    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:26.883692 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.883650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:26.883859 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.883706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:26.883859 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:26.883731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:26.883859 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.883841    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:26.883859 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.883849    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:26.884032 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.883858    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:26.884032 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.883889    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:26.884032 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.883900    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j72c5 for pod openshift-network-diagnostics/network-check-target-sfs7q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:26.884032 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.883909    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs podName:df547ee7-ed80-4d81-b8e0-b46339dae222 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:27.883888842 +0000 UTC m=+4.142088831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs") pod "network-metrics-daemon-trgt2" (UID: "df547ee7-ed80-4d81-b8e0-b46339dae222") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:26.884032 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.883930    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret podName:5fa3726c-9b76-4680-8d8b-0931a221c76f nodeName:}" failed. No retries permitted until 2026-09-11 13:48:27.883920558 +0000 UTC m=+4.142120545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret") pod "global-pull-secret-syncer-dg5cq" (UID: "5fa3726c-9b76-4680-8d8b-0931a221c76f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:26.884032 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:26.883944    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5 podName:5968c2e1-979b-4278-8e7a-dd70729bf698 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:27.883936739 +0000 UTC m=+4.142136727 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j72c5" (UniqueName: "kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5") pod "network-check-target-sfs7q" (UID: "5968c2e1-979b-4278-8e7a-dd70729bf698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:27.121297 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:27.121151    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12584014_59b3_4e2c_97e7_56c654c6bca6.slice/crio-42b0566bcde71fceed50c5d0f2f72b032a399a52e7189d947e5e004680610db4 WatchSource:0}: Error finding container 42b0566bcde71fceed50c5d0f2f72b032a399a52e7189d947e5e004680610db4: Status 404 returned error can't find the container with id 42b0566bcde71fceed50c5d0f2f72b032a399a52e7189d947e5e004680610db4
Sep 11 13:48:27.127390 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:27.127331    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78099fbe_6524_4477_8da7_1184017e0ed8.slice/crio-bac3cd02df1521eab7a8c95da0230c3336e961b675247364f4021d628f283306 WatchSource:0}: Error finding container bac3cd02df1521eab7a8c95da0230c3336e961b675247364f4021d628f283306: Status 404 returned error can't find the container with id bac3cd02df1521eab7a8c95da0230c3336e961b675247364f4021d628f283306
Sep 11 13:48:27.129841 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:27.129808    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb06779b_7d1e_4c26_b85c_1346bb5a1a01.slice/crio-458cdf4e9721e93906261f2141a6e7a9b4f0e4123eded9d16038a814037c619e WatchSource:0}: Error finding container 458cdf4e9721e93906261f2141a6e7a9b4f0e4123eded9d16038a814037c619e: Status 404 returned error can't find the container with id 458cdf4e9721e93906261f2141a6e7a9b4f0e4123eded9d16038a814037c619e
Sep 11 13:48:27.131138 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:27.130699    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefecc3fc_ad7a_4159_9c19_0d395f4c074c.slice/crio-5744fa867fdeea350bced2bd6824b19410b5af9631330152a6df01a243933f48 WatchSource:0}: Error finding container 5744fa867fdeea350bced2bd6824b19410b5af9631330152a6df01a243933f48: Status 404 returned error can't find the container with id 5744fa867fdeea350bced2bd6824b19410b5af9631330152a6df01a243933f48
Sep 11 13:48:27.131609 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:27.131585    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547be90d_fcb6_4db0_ae80_181de31c87b5.slice/crio-e98b29c97c4e940880da826bfa20bbd9e75c685e04a5bac9f92f1c5558df4bb4 WatchSource:0}: Error finding container e98b29c97c4e940880da826bfa20bbd9e75c685e04a5bac9f92f1c5558df4bb4: Status 404 returned error can't find the container with id e98b29c97c4e940880da826bfa20bbd9e75c685e04a5bac9f92f1c5558df4bb4
Sep 11 13:48:27.132541 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:27.132516    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75860117_a52f_47e3_8ab9_bc7d71df6bb1.slice/crio-b5c22e4a6f59375e43a77aa242f2eafc496a779c9a2e62960072a047c3aa2765 WatchSource:0}: Error finding container b5c22e4a6f59375e43a77aa242f2eafc496a779c9a2e62960072a047c3aa2765: Status 404 returned error can't find the container with id b5c22e4a6f59375e43a77aa242f2eafc496a779c9a2e62960072a047c3aa2765
Sep 11 13:48:27.135707 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:27.135682    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2adefd72_0d9f_40aa_a193_6b68449ce681.slice/crio-682157f30498b714000e737e7a827eb9e32ffc4a0e76d571df3d1fe96967be1f WatchSource:0}: Error finding container 682157f30498b714000e737e7a827eb9e32ffc4a0e76d571df3d1fe96967be1f: Status 404 returned error can't find the container with id 682157f30498b714000e737e7a827eb9e32ffc4a0e76d571df3d1fe96967be1f
Sep 11 13:48:27.207017 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.206966    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:43:26 +0000 UTC" deadline="2028-05-29 04:07:22.847447882 +0000 UTC"
Sep 11 13:48:27.207017 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.207017    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15014h18m55.640435466s"
Sep 11 13:48:27.243426 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.243395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"b5c22e4a6f59375e43a77aa242f2eafc496a779c9a2e62960072a047c3aa2765"}
Sep 11 13:48:27.244609 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.244571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh" event={"ID":"547be90d-fcb6-4db0-ae80-181de31c87b5","Type":"ContainerStarted","Data":"e98b29c97c4e940880da826bfa20bbd9e75c685e04a5bac9f92f1c5558df4bb4"}
Sep 11 13:48:27.246006 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.245833    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x4cs8" event={"ID":"db06779b-7d1e-4c26-b85c-1346bb5a1a01","Type":"ContainerStarted","Data":"458cdf4e9721e93906261f2141a6e7a9b4f0e4123eded9d16038a814037c619e"}
Sep 11 13:48:27.246982 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.246950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j6dmp" event={"ID":"efecc3fc-ad7a-4159-9c19-0d395f4c074c","Type":"ContainerStarted","Data":"5744fa867fdeea350bced2bd6824b19410b5af9631330152a6df01a243933f48"}
Sep 11 13:48:27.248627 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.248603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2tdnj" event={"ID":"78099fbe-6524-4477-8da7-1184017e0ed8","Type":"ContainerStarted","Data":"bac3cd02df1521eab7a8c95da0230c3336e961b675247364f4021d628f283306"}
Sep 11 13:48:27.250772 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.250704    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal" event={"ID":"f63e32dd172790f2b9a6ee29b1c6f139","Type":"ContainerStarted","Data":"9d3e25861dc3165ede24fa8463f237b54a2e81dd4b5337442a16f517a3eb2367"}
Sep 11 13:48:27.251983 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.251964    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4s6pt" event={"ID":"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b","Type":"ContainerStarted","Data":"1f4d7af4bb40cfabd42bfdbaaf15b1dc64ee13f3aba859e0ee167f61e0814aa3"}
Sep 11 13:48:27.253026 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.253006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerStarted","Data":"42b0566bcde71fceed50c5d0f2f72b032a399a52e7189d947e5e004680610db4"}
Sep 11 13:48:27.254226 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.254196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-454wg" event={"ID":"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77","Type":"ContainerStarted","Data":"56f789d0ef0136c9549a35546c7ed2c234888758a67786823f11e2152991d177"}
Sep 11 13:48:27.258675 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.258651    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nns9f" event={"ID":"2adefd72-0d9f-40aa-a193-6b68449ce681","Type":"ContainerStarted","Data":"682157f30498b714000e737e7a827eb9e32ffc4a0e76d571df3d1fe96967be1f"}
Sep 11 13:48:27.268490 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.268433    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-192.ec2.internal" podStartSLOduration=2.268413985 podStartE2EDuration="2.268413985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:27.267769033 +0000 UTC m=+3.525969034" watchObservedRunningTime="2026-09-11 13:48:27.268413985 +0000 UTC m=+3.526613994"
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.893372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.893431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:27.893479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:27.893628    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:27.893649    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:27.893662    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j72c5 for pod openshift-network-diagnostics/network-check-target-sfs7q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:27.893732    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5 podName:5968c2e1-979b-4278-8e7a-dd70729bf698 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:29.893712667 +0000 UTC m=+6.151912721 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j72c5" (UniqueName: "kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5") pod "network-check-target-sfs7q" (UID: "5968c2e1-979b-4278-8e7a-dd70729bf698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:27.893904    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:27.893984    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs podName:df547ee7-ed80-4d81-b8e0-b46339dae222 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:29.893955204 +0000 UTC m=+6.152155209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs") pod "network-metrics-daemon-trgt2" (UID: "df547ee7-ed80-4d81-b8e0-b46339dae222") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:27.894059    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:27.894109 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:27.894095    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret podName:5fa3726c-9b76-4680-8d8b-0931a221c76f nodeName:}" failed. No retries permitted until 2026-09-11 13:48:29.894084316 +0000 UTC m=+6.152284305 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret") pod "global-pull-secret-syncer-dg5cq" (UID: "5fa3726c-9b76-4680-8d8b-0931a221c76f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:28.207732 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:28.207666    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:43:26 +0000 UTC" deadline="2028-05-23 23:37:50.707617654 +0000 UTC"
Sep 11 13:48:28.207732 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:28.207730    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14889h49m22.499891714s"
Sep 11 13:48:28.234556 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:28.234065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:28.234556 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:28.234206    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:28.234556 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:28.234287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:28.234556 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:28.234394    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:28.234556 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:28.234510    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:28.234952 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:28.234609    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:28.270288 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:28.270255    2563 generic.go:356] "Generic (PLEG): container finished" podID="f66f75ce0b3c443239a1b57d427def08" containerID="548905a9abb6e9882f4b5b6492aa02936a9c97922c76617606b4417d0684595f" exitCode=0
Sep 11 13:48:28.270882 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:28.270806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal" event={"ID":"f66f75ce0b3c443239a1b57d427def08","Type":"ContainerDied","Data":"548905a9abb6e9882f4b5b6492aa02936a9c97922c76617606b4417d0684595f"}
Sep 11 13:48:29.297313 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:29.297280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal" event={"ID":"f66f75ce0b3c443239a1b57d427def08","Type":"ContainerStarted","Data":"0e4db44f52a76f55fb0eca019fd6f8211845338461afbef58f06d534b0dbe721"}
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:29.911803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:29.911909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:29.911981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:29.912119    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:29.912180    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs podName:df547ee7-ed80-4d81-b8e0-b46339dae222 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:33.912162418 +0000 UTC m=+10.170362411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs") pod "network-metrics-daemon-trgt2" (UID: "df547ee7-ed80-4d81-b8e0-b46339dae222") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:29.912683    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:29.912701    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:29.912713    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j72c5 for pod openshift-network-diagnostics/network-check-target-sfs7q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:29.912920 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:29.912862    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:29.913882 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:29.913817    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5 podName:5968c2e1-979b-4278-8e7a-dd70729bf698 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:33.912745074 +0000 UTC m=+10.170945063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j72c5" (UniqueName: "kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5") pod "network-check-target-sfs7q" (UID: "5968c2e1-979b-4278-8e7a-dd70729bf698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:29.913882 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:29.913861    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret podName:5fa3726c-9b76-4680-8d8b-0931a221c76f nodeName:}" failed. No retries permitted until 2026-09-11 13:48:33.913839602 +0000 UTC m=+10.172039597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret") pod "global-pull-secret-syncer-dg5cq" (UID: "5fa3726c-9b76-4680-8d8b-0931a221c76f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:30.244983 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:30.244901    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:30.245157 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:30.245076    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:30.245157 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:30.244909    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:30.245636 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:30.245611    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:30.245816 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:30.245638    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:30.245907 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:30.245834    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:32.237025 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:32.236995    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:32.237416 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:32.237123    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:32.237509 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:32.237489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:32.237636 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:32.237609    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:32.237710 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:32.237666    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:32.237864 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:32.237844    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:33.950184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:33.950299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:33.950353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:33.950479    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:33.950545    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret podName:5fa3726c-9b76-4680-8d8b-0931a221c76f nodeName:}" failed. No retries permitted until 2026-09-11 13:48:41.950527515 +0000 UTC m=+18.208727516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret") pod "global-pull-secret-syncer-dg5cq" (UID: "5fa3726c-9b76-4680-8d8b-0931a221c76f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:33.951090    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:33.951185    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs podName:df547ee7-ed80-4d81-b8e0-b46339dae222 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:41.951149171 +0000 UTC m=+18.209349174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs") pod "network-metrics-daemon-trgt2" (UID: "df547ee7-ed80-4d81-b8e0-b46339dae222") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:33.951268    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:33.951287    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:33.951299    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j72c5 for pod openshift-network-diagnostics/network-check-target-sfs7q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:33.951391 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:33.951351    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5 podName:5968c2e1-979b-4278-8e7a-dd70729bf698 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:41.951337935 +0000 UTC m=+18.209537937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j72c5" (UniqueName: "kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5") pod "network-check-target-sfs7q" (UID: "5968c2e1-979b-4278-8e7a-dd70729bf698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:34.235351 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:34.235270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:34.235552 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:34.235397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:34.235552 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:34.235408    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:34.235552 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:34.235528    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:34.235670 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:34.235581    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:34.235670 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:34.235632    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:36.234260 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:36.234220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:36.234735 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:36.234220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:36.234735 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:36.234359    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:36.234735 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:36.234221    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:36.234735 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:36.234433    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:36.234735 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:36.234497    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:38.234971 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:38.234678    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:38.235425 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:38.234682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:38.235425 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:38.235026    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:38.235425 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:38.234682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:38.235425 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:38.235110    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:38.235425 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:38.235170    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:40.234808 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:40.234722    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:40.235254 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:40.234722    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:40.235254 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:40.234883    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:40.235254 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:40.234873    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:40.235254 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:40.234957    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:40.235254 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:40.235036    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:42.018685 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:42.018652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:42.018706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:42.018738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.018847    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.018852    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.018874    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.018873    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.018888    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j72c5 for pod openshift-network-diagnostics/network-check-target-sfs7q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.018913    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs podName:df547ee7-ed80-4d81-b8e0-b46339dae222 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:58.018893144 +0000 UTC m=+34.277093145 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs") pod "network-metrics-daemon-trgt2" (UID: "df547ee7-ed80-4d81-b8e0-b46339dae222") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.018928    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5 podName:5968c2e1-979b-4278-8e7a-dd70729bf698 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:58.018918104 +0000 UTC m=+34.277118116 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j72c5" (UniqueName: "kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5") pod "network-check-target-sfs7q" (UID: "5968c2e1-979b-4278-8e7a-dd70729bf698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:42.019188 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.018947    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret podName:5fa3726c-9b76-4680-8d8b-0931a221c76f nodeName:}" failed. No retries permitted until 2026-09-11 13:48:58.018939279 +0000 UTC m=+34.277139266 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret") pod "global-pull-secret-syncer-dg5cq" (UID: "5fa3726c-9b76-4680-8d8b-0931a221c76f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:42.235025 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:42.234941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:42.235184 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.235081    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:42.235519 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:42.235500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:42.235626 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.235598    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:42.235769 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:42.235695    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:42.235838 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:42.235817    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:44.235541 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:44.235043    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:44.235541 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:44.235157    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:44.235541 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:44.235163    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:44.235541 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:44.235188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:44.235541 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:44.235244    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:44.235541 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:44.235316    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:46.234987 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.234745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:46.235638 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.234807    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:46.235638 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:46.235071    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:46.235638 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.234842    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:46.235638 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:46.235134    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:46.235638 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:46.235225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:46.335972 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.335935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2tdnj" event={"ID":"78099fbe-6524-4477-8da7-1184017e0ed8","Type":"ContainerStarted","Data":"5c8e2000a4ed637d8958f790ee33febc49f65a1fd07a6c313f34b39d74b93eec"}
Sep 11 13:48:46.337744 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.337717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4s6pt" event={"ID":"6cf760d1-d0fd-44cd-9a0e-8df9b5d60d2b","Type":"ContainerStarted","Data":"6df3b06a2dcf67c8de320232b7a9ba019a17964cae56ce85f7af106382ce2cda"}
Sep 11 13:48:46.339414 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.339337    2563 generic.go:356] "Generic (PLEG): container finished" podID="12584014-59b3-4e2c-97e7-56c654c6bca6" containerID="c9a85268dcd3beb5f7d1969d03a99981728dbd842f5a4abdd88addd05c601a39" exitCode=0
Sep 11 13:48:46.339414 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.339423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerDied","Data":"c9a85268dcd3beb5f7d1969d03a99981728dbd842f5a4abdd88addd05c601a39"}
Sep 11 13:48:46.341190 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.341165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nns9f" event={"ID":"2adefd72-0d9f-40aa-a193-6b68449ce681","Type":"ContainerStarted","Data":"1d30fb627437366c0b837b282d5d65eb2332d0654e02deb8281461b74fa1a782"}
Sep 11 13:48:46.344219 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.344196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"61dfb2a0d503a7843ca8fd45ce97fcc9f021735ebece122943dfdd994eab0dc2"}
Sep 11 13:48:46.344323 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.344286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"1af5281a040c07414f4dcf5bb3f383bb92413929dd4477567cf3397df4bbb35d"}
Sep 11 13:48:46.344386 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.344338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"830061bb5009df7e111eb6308ea0bd33256564f467764125d6e023e337f37ffb"}
Sep 11 13:48:46.345932 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.345911    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh" event={"ID":"547be90d-fcb6-4db0-ae80-181de31c87b5","Type":"ContainerStarted","Data":"34ca6b67512d5886e8457b713c781972eade0d73b733581a86cf0453aa44f430"}
Sep 11 13:48:46.347367 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.347343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x4cs8" event={"ID":"db06779b-7d1e-4c26-b85c-1346bb5a1a01","Type":"ContainerStarted","Data":"2c42f450cceb0f7dbb440e07d968c573f8ff99fd0c7f777867fa46c4267a06d2"}
Sep 11 13:48:46.348991 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.348967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j6dmp" event={"ID":"efecc3fc-ad7a-4159-9c19-0d395f4c074c","Type":"ContainerStarted","Data":"3c1e2c94fd5dfb4127e568cfba7e8daeb0eb1e0f1bad6add5fab5788752a0ab0"}
Sep 11 13:48:46.352160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.352122    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-192.ec2.internal" podStartSLOduration=21.352110104 podStartE2EDuration="21.352110104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:29.314088442 +0000 UTC m=+5.572288452" watchObservedRunningTime="2026-09-11 13:48:46.352110104 +0000 UTC m=+22.610310113"
Sep 11 13:48:46.352345 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.352327    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2tdnj" podStartSLOduration=3.733934913 podStartE2EDuration="22.352322085s" totalImagesPullingTime="18.618387172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:46.351915694 +0000 UTC m=+22.610115702" watchObservedRunningTime="2026-09-11 13:48:46.352322085 +0000 UTC m=+22.610522094"
Sep 11 13:48:46.368647 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.368597    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x4cs8" podStartSLOduration=3.752435319 podStartE2EDuration="22.368581335s" totalImagesPullingTime="18.616146016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:46.368438423 +0000 UTC m=+22.626638432" watchObservedRunningTime="2026-09-11 13:48:46.368581335 +0000 UTC m=+22.626781344"
Sep 11 13:48:46.384904 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.384855    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4s6pt" podStartSLOduration=3.694553891 podStartE2EDuration="22.384841508s" totalImagesPullingTime="18.690287617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:46.384018879 +0000 UTC m=+22.642218888" watchObservedRunningTime="2026-09-11 13:48:46.384841508 +0000 UTC m=+22.643041517"
Sep 11 13:48:46.417931 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.417876    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j6dmp" podStartSLOduration=3.80278538 podStartE2EDuration="22.417859613s" totalImagesPullingTime="18.615074233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:46.417833858 +0000 UTC m=+22.676033867" watchObservedRunningTime="2026-09-11 13:48:46.417859613 +0000 UTC m=+22.676059622"
Sep 11 13:48:46.432773 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.432702    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nns9f" podStartSLOduration=3.7952570249999997 podStartE2EDuration="22.432679903s" totalImagesPullingTime="18.637422878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:46.431789642 +0000 UTC m=+22.689989645" watchObservedRunningTime="2026-09-11 13:48:46.432679903 +0000 UTC m=+22.690879913"
Sep 11 13:48:46.467091 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.467059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:46.467547 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:46.467531    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:47.357652 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:47.357611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"7fc120362065963e64d34f1bfecbfd5752c462ac6a8d448dee2dd2560442e574"}
Sep 11 13:48:47.358236 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:47.357776    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"fc1985e05755dbc45115a2dfdbd8611b1a208d75b4492a46c9f07c0ea939d247"}
Sep 11 13:48:47.358429 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:47.358405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"2a76073ac3487f468304b4b6ca11f653679cd5f3a41262c24bd462779b22b56a"}
Sep 11 13:48:47.360307 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:47.359952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-454wg" event={"ID":"6fd2e0cf-2b37-4c09-a2cb-275a77bbcd77","Type":"ContainerStarted","Data":"e679cf74c00e7f7b308a44d1813e6d289321a3ee6980d9c7b313985fba279e6a"}
Sep 11 13:48:47.360954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:47.360925    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:47.361637 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:47.361613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x4cs8"
Sep 11 13:48:47.375793 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:47.375716    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-454wg" podStartSLOduration=4.720182606 podStartE2EDuration="23.375695985s" totalImagesPullingTime="18.655513379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:47.375136935 +0000 UTC m=+23.633336946" watchObservedRunningTime="2026-09-11 13:48:47.375695985 +0000 UTC m=+23.633895993"
Sep 11 13:48:47.488466 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:47.488438    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:48:48.235508 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:48.235001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:48.235508 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:48.234999    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:48.235508 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:48.235140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:48.235508 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:48.235136    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:48.235508 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:48.235225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:48.235508 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:48.235303    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:48.262255 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:48.262158    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:48:47.488459901Z","UUID":"22d2faf7-d024-466a-810c-627763e4376c","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:48:48.264030 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:48.264002    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:48:48.264178 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:48.264039    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:48:48.366190 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:48.366118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh" event={"ID":"547be90d-fcb6-4db0-ae80-181de31c87b5","Type":"ContainerStarted","Data":"dcb289d2de59707116761cf21b4df386aabdf914dc39fe2216eb2d72a04f9794"}
Sep 11 13:48:49.371485 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:49.371450    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"1c7a5918b3a594200ea9583a9410ce580738d60f5c08bd490e2714cdc3149c88"}
Sep 11 13:48:49.373841 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:49.373741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh" event={"ID":"547be90d-fcb6-4db0-ae80-181de31c87b5","Type":"ContainerStarted","Data":"c6fb16573a1fecbf0878b90a4caa3db9101c9dda9b0c2d6f5c04aa0e4e312eb5"}
Sep 11 13:48:49.391663 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:49.391612    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmlkh" podStartSLOduration=4.416625647 podStartE2EDuration="25.39159762s" totalImagesPullingTime="20.974971973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:49.390113638 +0000 UTC m=+25.648313646" watchObservedRunningTime="2026-09-11 13:48:49.39159762 +0000 UTC m=+25.649797639"
Sep 11 13:48:50.234145 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:50.234103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:50.234331 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:50.234226    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:50.234331 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:50.234103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:50.234331 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:50.234325    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:50.234481 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:50.234103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:50.234481 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:50.234391    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:51.384787 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:51.382588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" event={"ID":"75860117-a52f-47e3-8ab9-bc7d71df6bb1","Type":"ContainerStarted","Data":"897a378e551f8dd795dc1bf0206532896c3de4e7e2e46048077b2b2ad8dd1e29"}
Sep 11 13:48:51.384787 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:51.382892    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:51.384787 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:51.382943    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:51.384787 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:51.384711    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerStarted","Data":"9f891e52f21713818621bf32f69c1b9cbc1e8c5fdd303a9c194a1c21a70304fd"}
Sep 11 13:48:51.399362 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:51.399332    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:51.411583 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:51.411529    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f" podStartSLOduration=8.709444973 podStartE2EDuration="27.411510655s" totalImagesPullingTime="18.702065682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:51.410951933 +0000 UTC m=+27.669151942" watchObservedRunningTime="2026-09-11 13:48:51.411510655 +0000 UTC m=+27.669710663"
Sep 11 13:48:52.234311 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:52.234271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:52.234560 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:52.234272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:52.234560 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:52.234395    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:52.234560 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:52.234271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:52.234560 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:52.234465    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:52.234560 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:52.234525    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:52.388715 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:52.388681    2563 generic.go:356] "Generic (PLEG): container finished" podID="12584014-59b3-4e2c-97e7-56c654c6bca6" containerID="9f891e52f21713818621bf32f69c1b9cbc1e8c5fdd303a9c194a1c21a70304fd" exitCode=0
Sep 11 13:48:52.389174 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:52.388778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerDied","Data":"9f891e52f21713818621bf32f69c1b9cbc1e8c5fdd303a9c194a1c21a70304fd"}
Sep 11 13:48:52.389240 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:52.389218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:52.406135 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:52.406112    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:48:53.112151 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:53.112118    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sfs7q"]
Sep 11 13:48:53.112334 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:53.112239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:53.112399 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:53.112344    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:53.117220 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:53.117187    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dg5cq"]
Sep 11 13:48:53.117410 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:53.117307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:53.117469 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:53.117410    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:53.120511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:53.120451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-trgt2"]
Sep 11 13:48:53.120654 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:53.120561    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:53.120706 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:53.120670    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:54.395746 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:54.395714    2563 generic.go:356] "Generic (PLEG): container finished" podID="12584014-59b3-4e2c-97e7-56c654c6bca6" containerID="33eb9b4a9af6453dc05794c805fccad54e9aacad9f889cc7ab272fae9829cef4" exitCode=0
Sep 11 13:48:54.396317 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:54.395803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerDied","Data":"33eb9b4a9af6453dc05794c805fccad54e9aacad9f889cc7ab272fae9829cef4"}
Sep 11 13:48:55.234321 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:55.234287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:55.234321 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:55.234315    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:55.234515 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:55.234287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:55.234515 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:55.234415    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:55.234515 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:55.234447    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:55.234612 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:55.234537    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:55.401290 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:55.401023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerStarted","Data":"fa1e416dfa58af6ff73648397a9ea645c5f0fc3838225e50516e86de8045b798"}
Sep 11 13:48:56.405289 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:56.405255    2563 generic.go:356] "Generic (PLEG): container finished" podID="12584014-59b3-4e2c-97e7-56c654c6bca6" containerID="fa1e416dfa58af6ff73648397a9ea645c5f0fc3838225e50516e86de8045b798" exitCode=0
Sep 11 13:48:56.405735 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:56.405322    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerDied","Data":"fa1e416dfa58af6ff73648397a9ea645c5f0fc3838225e50516e86de8045b798"}
Sep 11 13:48:57.234438 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:57.234399    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:57.234438 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:57.234422    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:57.234695 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:57.234519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:57.234695 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:57.234529    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfs7q" podUID="5968c2e1-979b-4278-8e7a-dd70729bf698"
Sep 11 13:48:57.234695 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:57.234591    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trgt2" podUID="df547ee7-ed80-4d81-b8e0-b46339dae222"
Sep 11 13:48:57.234695 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:57.234664    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dg5cq" podUID="5fa3726c-9b76-4680-8d8b-0931a221c76f"
Sep 11 13:48:58.031328 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.031290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:58.031328 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.031339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.031378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.031476    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.031473    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.031494    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.031513    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j72c5 for pod openshift-network-diagnostics/network-check-target-sfs7q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.031516    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.031567    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5 podName:5968c2e1-979b-4278-8e7a-dd70729bf698 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:30.031545049 +0000 UTC m=+66.289745036 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j72c5" (UniqueName: "kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5") pod "network-check-target-sfs7q" (UID: "5968c2e1-979b-4278-8e7a-dd70729bf698") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.031587    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret podName:5fa3726c-9b76-4680-8d8b-0931a221c76f nodeName:}" failed. No retries permitted until 2026-09-11 13:49:30.031576898 +0000 UTC m=+66.289776885 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret") pod "global-pull-secret-syncer-dg5cq" (UID: "5fa3726c-9b76-4680-8d8b-0931a221c76f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:58.031944 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.031609    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs podName:df547ee7-ed80-4d81-b8e0-b46339dae222 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:30.031599572 +0000 UTC m=+66.289799559 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs") pod "network-metrics-daemon-trgt2" (UID: "df547ee7-ed80-4d81-b8e0-b46339dae222") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:58.600482 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.600454    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-192.ec2.internal" event="NodeReady"
Sep 11 13:48:58.600662 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.600598    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:48:58.639415 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.639381    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d569f65dd-ltwkl"]
Sep 11 13:48:58.643132 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.643115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.646059 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.646034    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 13:48:58.646183 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.646047    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 13:48:58.646505 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.646486    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8k7zz\""
Sep 11 13:48:58.646564 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.646549    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 13:48:58.654711 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.654121    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 13:48:58.655831 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.655783    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d569f65dd-ltwkl"]
Sep 11 13:48:58.659486 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.659460    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hkkx6"]
Sep 11 13:48:58.663082 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.663061    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7x4q2"]
Sep 11 13:48:58.663226 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.663207    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.666030 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.666006    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q92jh\""
Sep 11 13:48:58.666134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.666052    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:48:58.666197 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.666171    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:48:58.666626 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.666607    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7x4q2"
Sep 11 13:48:58.669270 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.669132    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xnqjg\""
Sep 11 13:48:58.669528 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.669509    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:48:58.669742 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.669724    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:48:58.670098 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.670014    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:48:58.671604 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.671580    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7x4q2"]
Sep 11 13:48:58.671701 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.671635    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hkkx6"]
Sep 11 13:48:58.737139 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mw9b\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-kube-api-access-7mw9b\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.737327 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-ca-trust-extracted\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.737327 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.737327 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-certificates\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.737327 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-config-volume\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.737327 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3-cert\") pod \"ingress-canary-7x4q2\" (UID: \"2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3\") " pod="openshift-ingress-canary/ingress-canary-7x4q2"
Sep 11 13:48:58.737327 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-installation-pull-secrets\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.737565 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck6n4\" (UniqueName: \"kubernetes.io/projected/2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3-kube-api-access-ck6n4\") pod \"ingress-canary-7x4q2\" (UID: \"2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3\") " pod="openshift-ingress-canary/ingress-canary-7x4q2"
Sep 11 13:48:58.737565 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-bound-sa-token\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.737565 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxpw7\" (UniqueName: \"kubernetes.io/projected/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-kube-api-access-mxpw7\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.737565 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.737565 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-trusted-ca\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.737823 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-tmp-dir\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.737823 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.737630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-image-registry-private-configuration\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.792034 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.791993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x4cs8_db06779b-7d1e-4c26-b85c-1346bb5a1a01/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:58.839007 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.838974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-image-registry-private-configuration\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839007 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mw9b\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-kube-api-access-7mw9b\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839238 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-ca-trust-extracted\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839238 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.839238 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-certificates\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839238 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.839169    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:48:58.839419 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.839251    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls podName:4b97da6d-72f4-4fa8-ae6f-6f8d2f325550 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:59.339230536 +0000 UTC m=+35.597430538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls") pod "dns-default-hkkx6" (UID: "4b97da6d-72f4-4fa8-ae6f-6f8d2f325550") : secret "dns-default-metrics-tls" not found
Sep 11 13:48:58.839419 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-config-volume\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.839419 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3-cert\") pod \"ingress-canary-7x4q2\" (UID: \"2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3\") " pod="openshift-ingress-canary/ingress-canary-7x4q2"
Sep 11 13:48:58.839419 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-installation-pull-secrets\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839419 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck6n4\" (UniqueName: \"kubernetes.io/projected/2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3-kube-api-access-ck6n4\") pod \"ingress-canary-7x4q2\" (UID: \"2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3\") " pod="openshift-ingress-canary/ingress-canary-7x4q2"
Sep 11 13:48:58.839419 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-bound-sa-token\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839719 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxpw7\" (UniqueName: \"kubernetes.io/projected/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-kube-api-access-mxpw7\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.839719 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839719 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-trusted-ca\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839719 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-tmp-dir\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.839961 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-certificates\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839961 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-tmp-dir\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.839961 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.839491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-ca-trust-extracted\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.839961 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.839944    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:48:58.839961 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.839962    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d569f65dd-ltwkl: secret "image-registry-tls" not found
Sep 11 13:48:58.840210 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:58.840016    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls podName:b93ab15a-57e1-443e-9fa0-a6f84b402b9d nodeName:}" failed. No retries permitted until 2026-09-11 13:48:59.34000151 +0000 UTC m=+35.598201512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls") pod "image-registry-7d569f65dd-ltwkl" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d") : secret "image-registry-tls" not found
Sep 11 13:48:58.840210 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.840040    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-config-volume\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.840738 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.840713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-trusted-ca\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.844095 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.844073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3-cert\") pod \"ingress-canary-7x4q2\" (UID: \"2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3\") " pod="openshift-ingress-canary/ingress-canary-7x4q2"
Sep 11 13:48:58.844095 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.844083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-image-registry-private-configuration\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.845951 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.845933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-installation-pull-secrets\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.852308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.852246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-bound-sa-token\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.852308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.852273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck6n4\" (UniqueName: \"kubernetes.io/projected/2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3-kube-api-access-ck6n4\") pod \"ingress-canary-7x4q2\" (UID: \"2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3\") " pod="openshift-ingress-canary/ingress-canary-7x4q2"
Sep 11 13:48:58.852717 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.852701    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mw9b\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-kube-api-access-7mw9b\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:58.852865 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.852845    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxpw7\" (UniqueName: \"kubernetes.io/projected/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-kube-api-access-mxpw7\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:58.881133 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.881090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x4cs8_db06779b-7d1e-4c26-b85c-1346bb5a1a01/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:58.981580 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:58.981545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7x4q2"
Sep 11 13:48:59.234140 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.234053    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:48:59.234592 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.234179    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:48:59.234592 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.234212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:48:59.238563 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.238539    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:48:59.238840 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.238822    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6pjch\""
Sep 11 13:48:59.238925 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.238856    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:48:59.239043 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.239024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:48:59.239606 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.239588    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:48:59.239716 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.239666    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlc2t\""
Sep 11 13:48:59.277577 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.277316    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7x4q2"]
Sep 11 13:48:59.283318 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:48:59.283288    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf035ca_5e6b_47ee_9bb2_9f66aa68c6f3.slice/crio-7c1ded85f06d9995943c8a81390d0ba6e401df7fcacccdad9d265ae1cf86a6b9 WatchSource:0}: Error finding container 7c1ded85f06d9995943c8a81390d0ba6e401df7fcacccdad9d265ae1cf86a6b9: Status 404 returned error can't find the container with id 7c1ded85f06d9995943c8a81390d0ba6e401df7fcacccdad9d265ae1cf86a6b9
Sep 11 13:48:59.343259 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.343232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:48:59.343423 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.343300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:48:59.343423 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:59.343420    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:48:59.343561 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:59.343432    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d569f65dd-ltwkl: secret "image-registry-tls" not found
Sep 11 13:48:59.343561 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:59.343445    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:48:59.343561 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:59.343482    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls podName:b93ab15a-57e1-443e-9fa0-a6f84b402b9d nodeName:}" failed. No retries permitted until 2026-09-11 13:49:00.343467956 +0000 UTC m=+36.601667943 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls") pod "image-registry-7d569f65dd-ltwkl" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d") : secret "image-registry-tls" not found
Sep 11 13:48:59.343561 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:48:59.343509    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls podName:4b97da6d-72f4-4fa8-ae6f-6f8d2f325550 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:00.343489198 +0000 UTC m=+36.601689200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls") pod "dns-default-hkkx6" (UID: "4b97da6d-72f4-4fa8-ae6f-6f8d2f325550") : secret "dns-default-metrics-tls" not found
Sep 11 13:48:59.413865 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:48:59.413820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7x4q2" event={"ID":"2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3","Type":"ContainerStarted","Data":"7c1ded85f06d9995943c8a81390d0ba6e401df7fcacccdad9d265ae1cf86a6b9"}
Sep 11 13:49:00.351213 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:00.351169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:00.351648 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:00.351247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:00.351648 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:00.351373    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:00.351648 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:00.351374    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:00.351648 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:00.351390    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d569f65dd-ltwkl: secret "image-registry-tls" not found
Sep 11 13:49:00.351648 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:00.351474    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls podName:4b97da6d-72f4-4fa8-ae6f-6f8d2f325550 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:02.351451526 +0000 UTC m=+38.609651530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls") pod "dns-default-hkkx6" (UID: "4b97da6d-72f4-4fa8-ae6f-6f8d2f325550") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:00.351648 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:00.351495    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls podName:b93ab15a-57e1-443e-9fa0-a6f84b402b9d nodeName:}" failed. No retries permitted until 2026-09-11 13:49:02.351484451 +0000 UTC m=+38.609684443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls") pod "image-registry-7d569f65dd-ltwkl" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d") : secret "image-registry-tls" not found
Sep 11 13:49:01.015570 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:01.015483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x4cs8_db06779b-7d1e-4c26-b85c-1346bb5a1a01/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:01.117701 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:01.117604    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x4cs8_db06779b-7d1e-4c26-b85c-1346bb5a1a01/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:01.423720 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:01.423631    2563 generic.go:356] "Generic (PLEG): container finished" podID="12584014-59b3-4e2c-97e7-56c654c6bca6" containerID="d89142e7df42c879b27454ae3c47b6316ce11a3cde61583e482391732ea95b73" exitCode=0
Sep 11 13:49:01.423720 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:01.423673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerDied","Data":"d89142e7df42c879b27454ae3c47b6316ce11a3cde61583e482391732ea95b73"}
Sep 11 13:49:02.367737 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:02.367698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:02.367937 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:02.367782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:02.367937 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:02.367889    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:02.367937 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:02.367926    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:02.368090 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:02.367940    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d569f65dd-ltwkl: secret "image-registry-tls" not found
Sep 11 13:49:02.368090 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:02.367973    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls podName:4b97da6d-72f4-4fa8-ae6f-6f8d2f325550 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:06.367950539 +0000 UTC m=+42.626150751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls") pod "dns-default-hkkx6" (UID: "4b97da6d-72f4-4fa8-ae6f-6f8d2f325550") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:02.368090 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:02.367991    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls podName:b93ab15a-57e1-443e-9fa0-a6f84b402b9d nodeName:}" failed. No retries permitted until 2026-09-11 13:49:06.367984604 +0000 UTC m=+42.626184591 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls") pod "image-registry-7d569f65dd-ltwkl" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d") : secret "image-registry-tls" not found
Sep 11 13:49:02.430534 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:02.430500    2563 generic.go:356] "Generic (PLEG): container finished" podID="12584014-59b3-4e2c-97e7-56c654c6bca6" containerID="0b419441ea2d3a76b4772a8c5b5e62aa06d6988c2f11b20430d495731db257cf" exitCode=0
Sep 11 13:49:02.430534 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:02.430549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerDied","Data":"0b419441ea2d3a76b4772a8c5b5e62aa06d6988c2f11b20430d495731db257cf"}
Sep 11 13:49:03.437160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:03.437105    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdlff" event={"ID":"12584014-59b3-4e2c-97e7-56c654c6bca6","Type":"ContainerStarted","Data":"5b2baa9a3343c678fe7323f908636a994c870daaf2e429beb244ace64ddc6455"}
Sep 11 13:49:03.438678 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:03.438652    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7x4q2" event={"ID":"2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3","Type":"ContainerStarted","Data":"e1538d945f7ce3d4b8b3cd8ad14b90988e0cddcc1e83dbd6824b5f991634b5db"}
Sep 11 13:49:03.461944 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:03.461881    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wdlff" podStartSLOduration=9.244162202 podStartE2EDuration="39.461861391s" totalImagesPullingTime="30.217699189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:03.460232174 +0000 UTC m=+39.718432184" watchObservedRunningTime="2026-09-11 13:49:03.461861391 +0000 UTC m=+39.720061403"
Sep 11 13:49:03.478127 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:03.478074    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7x4q2" podStartSLOduration=1.7121353369999999 podStartE2EDuration="5.478057653s" totalImagesPullingTime="3.765922316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:03.47681686 +0000 UTC m=+39.735016869" watchObservedRunningTime="2026-09-11 13:49:03.478057653 +0000 UTC m=+39.736257662"
Sep 11 13:49:06.396466 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:06.396417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:06.396919 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:06.396477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:06.396919 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:06.396578    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:06.396919 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:06.396589    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d569f65dd-ltwkl: secret "image-registry-tls" not found
Sep 11 13:49:06.396919 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:06.396598    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:06.396919 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:06.396640    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls podName:b93ab15a-57e1-443e-9fa0-a6f84b402b9d nodeName:}" failed. No retries permitted until 2026-09-11 13:49:14.396627634 +0000 UTC m=+50.654827633 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls") pod "image-registry-7d569f65dd-ltwkl" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d") : secret "image-registry-tls" not found
Sep 11 13:49:06.396919 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:06.396660    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls podName:4b97da6d-72f4-4fa8-ae6f-6f8d2f325550 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:14.396646183 +0000 UTC m=+50.654846170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls") pod "dns-default-hkkx6" (UID: "4b97da6d-72f4-4fa8-ae6f-6f8d2f325550") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:08.976648 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:08.976617    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"]
Sep 11 13:49:09.011725 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.011690    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wvb64"]
Sep 11 13:49:09.011928 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.011856    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:09.014663 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.014637    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-46bqg\""
Sep 11 13:49:09.014825 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.014637    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 13:49:09.014825 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.014637    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:49:09.014825 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.014637    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 13:49:09.024716 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.024690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"]
Sep 11 13:49:09.024846 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.024727    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wvb64"]
Sep 11 13:49:09.024846 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.024824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.027306 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.027282    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xtccf\""
Sep 11 13:49:09.027444 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.027291    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 13:49:09.027512 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.027317    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 13:49:09.027566 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.027391    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 13:49:09.027619 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.027569    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:49:09.032854 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.032830    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 13:49:09.077894 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.077863    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n"]
Sep 11 13:49:09.095195 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.095164    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"]
Sep 11 13:49:09.095377 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.095304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n"
Sep 11 13:49:09.099134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.099109    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-krnrz\""
Sep 11 13:49:09.099272 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.099112    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 13:49:09.099473 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.099458    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:49:09.105265 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.105245    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"]
Sep 11 13:49:09.106377 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.105369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.108349 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.108318    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qblkx\""
Sep 11 13:49:09.108522 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.108501    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:49:09.108699 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.108523    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 13:49:09.108699 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.108674    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 13:49:09.109115 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.109097    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 13:49:09.116576 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.116550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r555d\" (UniqueName: \"kubernetes.io/projected/541a0cd8-345c-4540-8fb2-7c108a798868-kube-api-access-r555d\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:09.116705 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.116588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e274ee6d-0a73-411d-b62a-e41022c1be0a-config\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.116705 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.116636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e274ee6d-0a73-411d-b62a-e41022c1be0a-serving-cert\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.116847 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.116711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:09.116847 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.116741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khphj\" (UniqueName: \"kubernetes.io/projected/e274ee6d-0a73-411d-b62a-e41022c1be0a-kube-api-access-khphj\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.116847 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.116828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e274ee6d-0a73-411d-b62a-e41022c1be0a-trusted-ca\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.119344 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.119320    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"]
Sep 11 13:49:09.119460 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.119436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.122367 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.122345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 13:49:09.122490 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.122373    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:49:09.122490 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.122354    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2gl89\""
Sep 11 13:49:09.122490 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.122475    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 13:49:09.122720 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.122704    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 13:49:09.131580 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.131552    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-565b697964-gmnzh"]
Sep 11 13:49:09.131687 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.131655    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.133994 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.133976    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 13:49:09.134247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.134227    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:49:09.134339 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.134282    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:49:09.135462 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.135448    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 13:49:09.135462 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.135458    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sqk69\""
Sep 11 13:49:09.143637 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.143616    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.143637 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.143614    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n"]
Sep 11 13:49:09.143827 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.143668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"]
Sep 11 13:49:09.143827 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.143684    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"]
Sep 11 13:49:09.143827 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.143737    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-565b697964-gmnzh"]
Sep 11 13:49:09.143827 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.143806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"]
Sep 11 13:49:09.147365 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.147345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 13:49:09.147496 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.147409    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 13:49:09.148107 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.148090    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 13:49:09.148205 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.148097    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:49:09.148701 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.148686    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:49:09.148803 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.148725    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kwsz9\""
Sep 11 13:49:09.148803 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.148787    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 13:49:09.207993 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.207955    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-w7d2b"]
Sep 11 13:49:09.213977 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.213946    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.217178 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.217147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e274ee6d-0a73-411d-b62a-e41022c1be0a-serving-cert\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.217568 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.217543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:09.217691 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.217597    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dlk6d\""
Sep 11 13:49:09.217691 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.217613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khphj\" (UniqueName: \"kubernetes.io/projected/e274ee6d-0a73-411d-b62a-e41022c1be0a-kube-api-access-khphj\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.217691 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.217668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p5p7\" (UniqueName: \"kubernetes.io/projected/b7f01e20-525e-49ae-98fd-4ca29745b391-kube-api-access-6p5p7\") pod \"volume-data-source-validator-86695b7cc7-hps9n\" (UID: \"b7f01e20-525e-49ae-98fd-4ca29745b391\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n"
Sep 11 13:49:09.217885 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.217712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-serving-cert\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.217885 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.217765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-config\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.217885 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.217847    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:49:09.218107 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.218091    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls podName:541a0cd8-345c-4540-8fb2-7c108a798868 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:09.718060422 +0000 UTC m=+45.976260424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-lwkxr" (UID: "541a0cd8-345c-4540-8fb2-7c108a798868") : secret "samples-operator-tls" not found
Sep 11 13:49:09.218282 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.218265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e274ee6d-0a73-411d-b62a-e41022c1be0a-trusted-ca\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.218419 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.218402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cht9t\" (UniqueName: \"kubernetes.io/projected/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-kube-api-access-cht9t\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.218527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.218511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r555d\" (UniqueName: \"kubernetes.io/projected/541a0cd8-345c-4540-8fb2-7c108a798868-kube-api-access-r555d\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:09.218816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.218617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e274ee6d-0a73-411d-b62a-e41022c1be0a-config\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.218954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.218937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/716a3a57-8906-41de-92c1-b0e0173338ad-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.219075 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.219060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/716a3a57-8906-41de-92c1-b0e0173338ad-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.219194 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.219178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vb76\" (UniqueName: \"kubernetes.io/projected/716a3a57-8906-41de-92c1-b0e0173338ad-kube-api-access-6vb76\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.219698 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.219670    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 13:49:09.219804 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.219732    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:49:09.220295 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.220274    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e274ee6d-0a73-411d-b62a-e41022c1be0a-config\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.220518 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.220483    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:49:09.220933 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.220900    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 13:49:09.221225 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.220937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e274ee6d-0a73-411d-b62a-e41022c1be0a-trusted-ca\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.223911 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.223325    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e274ee6d-0a73-411d-b62a-e41022c1be0a-serving-cert\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.226690 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.226639    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-sndmm"]
Sep 11 13:49:09.227797 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.227780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 13:49:09.231965 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.231945    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"]
Sep 11 13:49:09.232080 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.232065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-sndmm"
Sep 11 13:49:09.238015 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.237995    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:09.238361 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.238342    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cvr7f\""
Sep 11 13:49:09.240947 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.240929    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 13:49:09.241372 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.241353    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4lh6n\""
Sep 11 13:49:09.241644 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.241623    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 13:49:09.243516 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.243494    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-w7d2b"]
Sep 11 13:49:09.244429 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.244405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khphj\" (UniqueName: \"kubernetes.io/projected/e274ee6d-0a73-411d-b62a-e41022c1be0a-kube-api-access-khphj\") pod \"console-operator-696884fb9c-wvb64\" (UID: \"e274ee6d-0a73-411d-b62a-e41022c1be0a\") " pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.251300 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.251279    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-sndmm"]
Sep 11 13:49:09.252399 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.252379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"]
Sep 11 13:49:09.255052 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.255029    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r555d\" (UniqueName: \"kubernetes.io/projected/541a0cd8-345c-4540-8fb2-7c108a798868-kube-api-access-r555d\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:09.320696 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.320696 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6n6g\" (UniqueName: \"kubernetes.io/projected/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-kube-api-access-n6n6g\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p5p7\" (UniqueName: \"kubernetes.io/projected/b7f01e20-525e-49ae-98fd-4ca29745b391-kube-api-access-6p5p7\") pod \"volume-data-source-validator-86695b7cc7-hps9n\" (UID: \"b7f01e20-525e-49ae-98fd-4ca29745b391\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320782    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/716a3a57-8906-41de-92c1-b0e0173338ad-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vb76\" (UniqueName: \"kubernetes.io/projected/716a3a57-8906-41de-92c1-b0e0173338ad-kube-api-access-6vb76\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-tmp\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-stats-auth\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-config\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.320991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmk5d\" (UniqueName: \"kubernetes.io/projected/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-kube-api-access-vmk5d\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-serving-cert\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.321041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-service-ca-bundle\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.321511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-serving-cert\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.321511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-default-certificate\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.321511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.321511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-snapshots\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.321511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cht9t\" (UniqueName: \"kubernetes.io/projected/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-kube-api-access-cht9t\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.321511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/716a3a57-8906-41de-92c1-b0e0173338ad-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.321511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.321511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsbbk\" (UniqueName: \"kubernetes.io/projected/d967ed16-c979-45a0-aef2-4c4ccf73c12e-kube-api-access-qsbbk\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.321839 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/716a3a57-8906-41de-92c1-b0e0173338ad-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.321839 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.321816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-config\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.323334 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.323312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/716a3a57-8906-41de-92c1-b0e0173338ad-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.323485 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.323468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-serving-cert\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.331116 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.331082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cht9t\" (UniqueName: \"kubernetes.io/projected/5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303-kube-api-access-cht9t\") pod \"service-ca-operator-cf7c776b7-85r9s\" (UID: \"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.331272 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.331250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vb76\" (UniqueName: \"kubernetes.io/projected/716a3a57-8906-41de-92c1-b0e0173338ad-kube-api-access-6vb76\") pod \"kube-storage-version-migrator-operator-6c8f84f897-tgm76\" (UID: \"716a3a57-8906-41de-92c1-b0e0173338ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.331332 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.331278    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p5p7\" (UniqueName: \"kubernetes.io/projected/b7f01e20-525e-49ae-98fd-4ca29745b391-kube-api-access-6p5p7\") pod \"volume-data-source-validator-86695b7cc7-hps9n\" (UID: \"b7f01e20-525e-49ae-98fd-4ca29745b391\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n"
Sep 11 13:49:09.334136 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.334116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:09.406288 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.406258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n"
Sep 11 13:49:09.417430 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.417398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"
Sep 11 13:49:09.421828 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.421802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4b4a3162-901d-416a-8e12-569888476830-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:09.421960 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.421834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:09.421960 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.421905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.421960 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.421945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.422072 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.421979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-tmp\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.422072 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.421990    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:49:09.422072 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-stats-auth\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.422072 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.422035    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:09.922021141 +0000 UTC m=+46.180221133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : secret "router-metrics-certs-default" not found
Sep 11 13:49:09.422072 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmk5d\" (UniqueName: \"kubernetes.io/projected/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-kube-api-access-vmk5d\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.422247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-service-ca-bundle\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.422247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-serving-cert\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.422247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-default-certificate\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.422247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.422247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-snapshots\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.422247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd7fv\" (UniqueName: \"kubernetes.io/projected/b894314e-7ef1-4d5a-abf3-71cd0fe621e2-kube-api-access-xd7fv\") pod \"network-check-source-6f8b768455-sndmm\" (UID: \"b894314e-7ef1-4d5a-abf3-71cd0fe621e2\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-sndmm"
Sep 11 13:49:09.422247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.422247 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsbbk\" (UniqueName: \"kubernetes.io/projected/d967ed16-c979-45a0-aef2-4c4ccf73c12e-kube-api-access-qsbbk\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.422621 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.422621 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6n6g\" (UniqueName: \"kubernetes.io/projected/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-kube-api-access-n6n6g\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.422918 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.422888    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:09.423024 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.422961    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls podName:b373479a-c8a5-4ba1-9a2f-918bf37b97cb nodeName:}" failed. No retries permitted until 2026-09-11 13:49:09.922943458 +0000 UTC m=+46.181143465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hrqh9" (UID: "b373479a-c8a5-4ba1-9a2f-918bf37b97cb") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:09.423024 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.422966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-snapshots\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.423244 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.423223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-tmp\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.423352 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.423333    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:09.923221372 +0000 UTC m=+46.181421359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : configmap references non-existent config key: service-ca.crt
Sep 11 13:49:09.423687 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.423488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.424113 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.424092    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-service-ca-bundle\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.425149 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.425133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-stats-auth\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.425973 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.425953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-serving-cert\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.426064 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.426014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-default-certificate\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.427920 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.427902    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"
Sep 11 13:49:09.431187 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.431162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6n6g\" (UniqueName: \"kubernetes.io/projected/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-kube-api-access-n6n6g\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.431801 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.431780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsbbk\" (UniqueName: \"kubernetes.io/projected/d967ed16-c979-45a0-aef2-4c4ccf73c12e-kube-api-access-qsbbk\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.432684 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.432666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmk5d\" (UniqueName: \"kubernetes.io/projected/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-kube-api-access-vmk5d\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.434451 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.434431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c08c9a-f359-4fa7-b3a2-4b04500dd53e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-w7d2b\" (UID: \"59c08c9a-f359-4fa7-b3a2-4b04500dd53e\") " pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.525033 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.524997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd7fv\" (UniqueName: \"kubernetes.io/projected/b894314e-7ef1-4d5a-abf3-71cd0fe621e2-kube-api-access-xd7fv\") pod \"network-check-source-6f8b768455-sndmm\" (UID: \"b894314e-7ef1-4d5a-abf3-71cd0fe621e2\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-sndmm"
Sep 11 13:49:09.525156 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.525140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4b4a3162-901d-416a-8e12-569888476830-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:09.525208 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.525169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:09.525347 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.525296    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:49:09.525426 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.525364    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert podName:4b4a3162-901d-416a-8e12-569888476830 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:10.025343779 +0000 UTC m=+46.283543768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rw8gf" (UID: "4b4a3162-901d-416a-8e12-569888476830") : secret "networking-console-plugin-cert" not found
Sep 11 13:49:09.526305 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.526276    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4b4a3162-901d-416a-8e12-569888476830-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:09.530960 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.530934    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-w7d2b"
Sep 11 13:49:09.534686 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.534640    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd7fv\" (UniqueName: \"kubernetes.io/projected/b894314e-7ef1-4d5a-abf3-71cd0fe621e2-kube-api-access-xd7fv\") pod \"network-check-source-6f8b768455-sndmm\" (UID: \"b894314e-7ef1-4d5a-abf3-71cd0fe621e2\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-sndmm"
Sep 11 13:49:09.541216 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.541137    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-sndmm"
Sep 11 13:49:09.626325 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.626297    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wvb64"]
Sep 11 13:49:09.629980 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:09.629946    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode274ee6d_0a73_411d_b62a_e41022c1be0a.slice/crio-3de439c5fa441ec0f90cd37fecafd0bcefdb65ca9e374368729f8e25910e6b69 WatchSource:0}: Error finding container 3de439c5fa441ec0f90cd37fecafd0bcefdb65ca9e374368729f8e25910e6b69: Status 404 returned error can't find the container with id 3de439c5fa441ec0f90cd37fecafd0bcefdb65ca9e374368729f8e25910e6b69
Sep 11 13:49:09.698354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.698229    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n"]
Sep 11 13:49:09.700839 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:09.700808    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f01e20_525e_49ae_98fd_4ca29745b391.slice/crio-e0b62677fb2aa06bd24f62d2b9354e212b732e8ee65a2418922ba0943c598014 WatchSource:0}: Error finding container e0b62677fb2aa06bd24f62d2b9354e212b732e8ee65a2418922ba0943c598014: Status 404 returned error can't find the container with id e0b62677fb2aa06bd24f62d2b9354e212b732e8ee65a2418922ba0943c598014
Sep 11 13:49:09.726446 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.726417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:09.726668 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.726570    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:49:09.726668 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.726643    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls podName:541a0cd8-345c-4540-8fb2-7c108a798868 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:10.726624312 +0000 UTC m=+46.984824313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-lwkxr" (UID: "541a0cd8-345c-4540-8fb2-7c108a798868") : secret "samples-operator-tls" not found
Sep 11 13:49:09.733678 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.733603    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s"]
Sep 11 13:49:09.736785 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:09.736763    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5641e3ee_4ac9_47ed_ae8c_5cdb7dee2303.slice/crio-e9e95a50eb1362508dc33a011884dd6234e2348a9a7196b99bab45f6bd66af49 WatchSource:0}: Error finding container e9e95a50eb1362508dc33a011884dd6234e2348a9a7196b99bab45f6bd66af49: Status 404 returned error can't find the container with id e9e95a50eb1362508dc33a011884dd6234e2348a9a7196b99bab45f6bd66af49
Sep 11 13:49:09.764300 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.764267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76"]
Sep 11 13:49:09.768859 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:09.768828    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716a3a57_8906_41de_92c1_b0e0173338ad.slice/crio-9be409a736e8fc3684b64fd41253a8f23e4783bfa49f3be0440c798ce2da8254 WatchSource:0}: Error finding container 9be409a736e8fc3684b64fd41253a8f23e4783bfa49f3be0440c798ce2da8254: Status 404 returned error can't find the container with id 9be409a736e8fc3684b64fd41253a8f23e4783bfa49f3be0440c798ce2da8254
Sep 11 13:49:09.818120 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.818089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-w7d2b"]
Sep 11 13:49:09.820933 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:09.820904    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c08c9a_f359_4fa7_b3a2_4b04500dd53e.slice/crio-82917870b1e4f6c40909d9deca51de5e495fd955a6eed9310cc32ed971a36bb1 WatchSource:0}: Error finding container 82917870b1e4f6c40909d9deca51de5e495fd955a6eed9310cc32ed971a36bb1: Status 404 returned error can't find the container with id 82917870b1e4f6c40909d9deca51de5e495fd955a6eed9310cc32ed971a36bb1
Sep 11 13:49:09.839527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.839500    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-sndmm"]
Sep 11 13:49:09.842898 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:09.842873    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb894314e_7ef1_4d5a_abf3_71cd0fe621e2.slice/crio-765ab46e43b97b38f66f07a530319cba36f21e385c7a3255f761a82ded662654 WatchSource:0}: Error finding container 765ab46e43b97b38f66f07a530319cba36f21e385c7a3255f761a82ded662654: Status 404 returned error can't find the container with id 765ab46e43b97b38f66f07a530319cba36f21e385c7a3255f761a82ded662654
Sep 11 13:49:09.928619 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.928574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:09.928862 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.928649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.928862 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:09.928705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:09.928862 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.928744    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:09.928862 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.928833    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls podName:b373479a-c8a5-4ba1-9a2f-918bf37b97cb nodeName:}" failed. No retries permitted until 2026-09-11 13:49:10.928815386 +0000 UTC m=+47.187015395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hrqh9" (UID: "b373479a-c8a5-4ba1-9a2f-918bf37b97cb") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:09.928862 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.928854    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:10.928839201 +0000 UTC m=+47.187039188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : configmap references non-existent config key: service-ca.crt
Sep 11 13:49:09.929074 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.928868    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:49:09.929074 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:09.928920    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:10.928906008 +0000 UTC m=+47.187106011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : secret "router-metrics-certs-default" not found
Sep 11 13:49:10.033555 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.030446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:10.033555 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.030788    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:49:10.033555 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.030906    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert podName:4b4a3162-901d-416a-8e12-569888476830 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:11.030841876 +0000 UTC m=+47.289041887 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rw8gf" (UID: "4b4a3162-901d-416a-8e12-569888476830") : secret "networking-console-plugin-cert" not found
Sep 11 13:49:10.461133 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.461042    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76" event={"ID":"716a3a57-8906-41de-92c1-b0e0173338ad","Type":"ContainerStarted","Data":"9be409a736e8fc3684b64fd41253a8f23e4783bfa49f3be0440c798ce2da8254"}
Sep 11 13:49:10.463023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.462947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wvb64" event={"ID":"e274ee6d-0a73-411d-b62a-e41022c1be0a","Type":"ContainerStarted","Data":"3de439c5fa441ec0f90cd37fecafd0bcefdb65ca9e374368729f8e25910e6b69"}
Sep 11 13:49:10.465427 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.465392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s" event={"ID":"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303","Type":"ContainerStarted","Data":"e9e95a50eb1362508dc33a011884dd6234e2348a9a7196b99bab45f6bd66af49"}
Sep 11 13:49:10.467663 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.467591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-w7d2b" event={"ID":"59c08c9a-f359-4fa7-b3a2-4b04500dd53e","Type":"ContainerStarted","Data":"82917870b1e4f6c40909d9deca51de5e495fd955a6eed9310cc32ed971a36bb1"}
Sep 11 13:49:10.469838 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.469800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-sndmm" event={"ID":"b894314e-7ef1-4d5a-abf3-71cd0fe621e2","Type":"ContainerStarted","Data":"765ab46e43b97b38f66f07a530319cba36f21e385c7a3255f761a82ded662654"}
Sep 11 13:49:10.473348 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.473316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n" event={"ID":"b7f01e20-525e-49ae-98fd-4ca29745b391","Type":"ContainerStarted","Data":"e0b62677fb2aa06bd24f62d2b9354e212b732e8ee65a2418922ba0943c598014"}
Sep 11 13:49:10.738488 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.737678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:10.738488 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.737889    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:49:10.738488 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.737958    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls podName:541a0cd8-345c-4540-8fb2-7c108a798868 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:12.73793803 +0000 UTC m=+48.996138031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-lwkxr" (UID: "541a0cd8-345c-4540-8fb2-7c108a798868") : secret "samples-operator-tls" not found
Sep 11 13:49:10.940272 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.940231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:10.940449 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.940320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:10.940516 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:10.940447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:10.940611 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.940593    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:10.940700 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.940660    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls podName:b373479a-c8a5-4ba1-9a2f-918bf37b97cb nodeName:}" failed. No retries permitted until 2026-09-11 13:49:12.940642504 +0000 UTC m=+49.198842506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hrqh9" (UID: "b373479a-c8a5-4ba1-9a2f-918bf37b97cb") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:10.941166 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.941144    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:12.941125965 +0000 UTC m=+49.199325966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : configmap references non-existent config key: service-ca.crt
Sep 11 13:49:10.941261 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.941224    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:49:10.941261 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:10.941257    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:12.941246217 +0000 UTC m=+49.199446219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : secret "router-metrics-certs-default" not found
Sep 11 13:49:11.041444 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:11.041347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:11.041876 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:11.041551    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:49:11.041876 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:11.041621    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert podName:4b4a3162-901d-416a-8e12-569888476830 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:13.041601799 +0000 UTC m=+49.299801793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rw8gf" (UID: "4b4a3162-901d-416a-8e12-569888476830") : secret "networking-console-plugin-cert" not found
Sep 11 13:49:12.758279 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:12.758181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:12.758745 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:12.758318    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:49:12.758745 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:12.758378    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls podName:541a0cd8-345c-4540-8fb2-7c108a798868 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:16.758364292 +0000 UTC m=+53.016564293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-lwkxr" (UID: "541a0cd8-345c-4540-8fb2-7c108a798868") : secret "samples-operator-tls" not found
Sep 11 13:49:12.959937 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:12.959894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:12.960112 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:12.960030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:12.960112 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:12.960065    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:49:12.960112 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:12.960089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:12.960233 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:12.960136    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:16.960116128 +0000 UTC m=+53.218316138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : secret "router-metrics-certs-default" not found
Sep 11 13:49:12.960233 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:12.960178    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:12.960233 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:12.960193    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:16.960182444 +0000 UTC m=+53.218382446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : configmap references non-existent config key: service-ca.crt
Sep 11 13:49:12.960341 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:12.960253    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls podName:b373479a-c8a5-4ba1-9a2f-918bf37b97cb nodeName:}" failed. No retries permitted until 2026-09-11 13:49:16.960237159 +0000 UTC m=+53.218437147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hrqh9" (UID: "b373479a-c8a5-4ba1-9a2f-918bf37b97cb") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:13.061481 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:13.061396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:13.061628 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:13.061562    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:49:13.061666 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:13.061628    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert podName:4b4a3162-901d-416a-8e12-569888476830 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:17.061611988 +0000 UTC m=+53.319811989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rw8gf" (UID: "4b4a3162-901d-416a-8e12-569888476830") : secret "networking-console-plugin-cert" not found
Sep 11 13:49:14.476094 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:14.476051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:14.476515 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:14.476121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:14.476515 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:14.476221    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:14.476515 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:14.476232    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d569f65dd-ltwkl: secret "image-registry-tls" not found
Sep 11 13:49:14.476515 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:14.476244    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:14.476515 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:14.476282    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls podName:b93ab15a-57e1-443e-9fa0-a6f84b402b9d nodeName:}" failed. No retries permitted until 2026-09-11 13:49:30.476268502 +0000 UTC m=+66.734468493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls") pod "image-registry-7d569f65dd-ltwkl" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d") : secret "image-registry-tls" not found
Sep 11 13:49:14.476515 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:14.476313    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls podName:4b97da6d-72f4-4fa8-ae6f-6f8d2f325550 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:30.476293948 +0000 UTC m=+66.734493950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls") pod "dns-default-hkkx6" (UID: "4b97da6d-72f4-4fa8-ae6f-6f8d2f325550") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:16.495574 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.495485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-w7d2b" event={"ID":"59c08c9a-f359-4fa7-b3a2-4b04500dd53e","Type":"ContainerStarted","Data":"249b53de905288c01da8c3d4e6cb10ca30d3a95005faa8a1f23735e5ea2a9f4d"}
Sep 11 13:49:16.497674 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.497643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-sndmm" event={"ID":"b894314e-7ef1-4d5a-abf3-71cd0fe621e2","Type":"ContainerStarted","Data":"39db9df5e986f6dbb64dc53e90acfbbecab866f385771a91a999e38ee2814ba6"}
Sep 11 13:49:16.500156 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.499791    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n" event={"ID":"b7f01e20-525e-49ae-98fd-4ca29745b391","Type":"ContainerStarted","Data":"be5ab3fc22fc0c46da45eeff919fe7fe9779c45f22c56d05d7718cbf1ec2820a"}
Sep 11 13:49:16.501622 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.501598    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76" event={"ID":"716a3a57-8906-41de-92c1-b0e0173338ad","Type":"ContainerStarted","Data":"caf0ab3a5df7290ceed7d1687d1b7fea2ae04c90c2302f21046dcb4fcc2f564c"}
Sep 11 13:49:16.503486 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.503466    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/0.log"
Sep 11 13:49:16.503603 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.503504    2563 generic.go:356] "Generic (PLEG): container finished" podID="e274ee6d-0a73-411d-b62a-e41022c1be0a" containerID="4854223a6b47203fe2ec7d247721c933e45916e43d2e9c29c5979be6922a4b5e" exitCode=255
Sep 11 13:49:16.503742 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.503716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wvb64" event={"ID":"e274ee6d-0a73-411d-b62a-e41022c1be0a","Type":"ContainerDied","Data":"4854223a6b47203fe2ec7d247721c933e45916e43d2e9c29c5979be6922a4b5e"}
Sep 11 13:49:16.503883 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.503846    2563 scope.go:117] "RemoveContainer" containerID="4854223a6b47203fe2ec7d247721c933e45916e43d2e9c29c5979be6922a4b5e"
Sep 11 13:49:16.505226 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.505183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s" event={"ID":"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303","Type":"ContainerStarted","Data":"171c7f134b5494e5a3576e5f929cf0a7853b7fdc45a952f5df76997ec6c99146"}
Sep 11 13:49:16.512854 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.512803    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-w7d2b" podStartSLOduration=1.385934254 podStartE2EDuration="7.51278438s" totalImagesPullingTime="6.126850126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:16.511669866 +0000 UTC m=+52.769869877" watchObservedRunningTime="2026-09-11 13:49:16.51278438 +0000 UTC m=+52.770984387"
Sep 11 13:49:16.562436 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.562376    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hps9n" podStartSLOduration=1.394182434 podStartE2EDuration="7.562351425s" totalImagesPullingTime="6.168168991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:16.56230086 +0000 UTC m=+52.820500870" watchObservedRunningTime="2026-09-11 13:49:16.562351425 +0000 UTC m=+52.820551436"
Sep 11 13:49:16.563207 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.563171    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-sndmm" podStartSLOduration=1.457735438 podStartE2EDuration="7.563156655s" totalImagesPullingTime="6.105421217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:16.543917611 +0000 UTC m=+52.802117622" watchObservedRunningTime="2026-09-11 13:49:16.563156655 +0000 UTC m=+52.821356666"
Sep 11 13:49:16.584537 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.584344    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s" podStartSLOduration=1.366995853 podStartE2EDuration="7.58432256s" totalImagesPullingTime="6.217326707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:16.582887287 +0000 UTC m=+52.841087300" watchObservedRunningTime="2026-09-11 13:49:16.58432256 +0000 UTC m=+52.842522570"
Sep 11 13:49:16.799607 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.799508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:16.799798 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:16.799690    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:49:16.799894 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:16.799806    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls podName:541a0cd8-345c-4540-8fb2-7c108a798868 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:24.799782302 +0000 UTC m=+61.057982292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-lwkxr" (UID: "541a0cd8-345c-4540-8fb2-7c108a798868") : secret "samples-operator-tls" not found
Sep 11 13:49:16.886403 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.886338    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76" podStartSLOduration=1.7069514030000001 podStartE2EDuration="7.886313742s" totalImagesPullingTime="6.179362339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:16.603031003 +0000 UTC m=+52.861231013" watchObservedRunningTime="2026-09-11 13:49:16.886313742 +0000 UTC m=+53.144513763"
Sep 11 13:49:16.888096 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.888069    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xdn8q"]
Sep 11 13:49:16.890778 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.890738    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:16.893828 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.893806    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-klbks\""
Sep 11 13:49:16.894353 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.894145    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:49:16.894353 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.894162    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:49:16.901589 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:16.901452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xdn8q"]
Sep 11 13:49:17.001545 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.001507    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.001732 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.001708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:17.001821 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.001786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.001870 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.001830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z2fh\" (UniqueName: \"kubernetes.io/projected/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-api-access-9z2fh\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.001870 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.001859    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:17.001957 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.001925    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls podName:b373479a-c8a5-4ba1-9a2f-918bf37b97cb nodeName:}" failed. No retries permitted until 2026-09-11 13:49:25.001905633 +0000 UTC m=+61.260105628 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hrqh9" (UID: "b373479a-c8a5-4ba1-9a2f-918bf37b97cb") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:17.001957 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.001860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:17.002042 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.001960    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:25.001945015 +0000 UTC m=+61.260145009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : configmap references non-existent config key: service-ca.crt
Sep 11 13:49:17.002042 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.001995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/89f217f3-bc97-424a-9084-27bf60ffcc6a-data-volume\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.002124 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.002055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:17.002124 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.002083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/89f217f3-bc97-424a-9084-27bf60ffcc6a-crio-socket\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.002207 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.002172    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:49:17.002238 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.002227    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:25.002211968 +0000 UTC m=+61.260411960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : secret "router-metrics-certs-default" not found
Sep 11 13:49:17.103156 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.103125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/89f217f3-bc97-424a-9084-27bf60ffcc6a-crio-socket\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.103381 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.103174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.103381 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.103266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/89f217f3-bc97-424a-9084-27bf60ffcc6a-crio-socket\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.103381 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.103273    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:17.103381 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.103348    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:17.603325486 +0000 UTC m=+53.861525474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:17.103537 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.103431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.103537 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.103470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z2fh\" (UniqueName: \"kubernetes.io/projected/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-api-access-9z2fh\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.103537 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.103512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/89f217f3-bc97-424a-9084-27bf60ffcc6a-data-volume\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.103537 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.103522    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:17.103716 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.103536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:17.103716 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.103573    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:17.603555918 +0000 UTC m=+53.861755920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:17.103716 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.103620    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:49:17.103716 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.103663    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert podName:4b4a3162-901d-416a-8e12-569888476830 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:25.103649009 +0000 UTC m=+61.361848998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rw8gf" (UID: "4b4a3162-901d-416a-8e12-569888476830") : secret "networking-console-plugin-cert" not found
Sep 11 13:49:17.104021 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.103997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/89f217f3-bc97-424a-9084-27bf60ffcc6a-data-volume\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.112271 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.112243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z2fh\" (UniqueName: \"kubernetes.io/projected/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-api-access-9z2fh\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.511413 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.511324    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 13:49:17.511838 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.511822    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/0.log"
Sep 11 13:49:17.511892 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.511862    2563 generic.go:356] "Generic (PLEG): container finished" podID="e274ee6d-0a73-411d-b62a-e41022c1be0a" containerID="02cbffd0c56b32111416a47d48182800fa8c4c3299967c067580b0a051223fe0" exitCode=255
Sep 11 13:49:17.511990 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.511966    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wvb64" event={"ID":"e274ee6d-0a73-411d-b62a-e41022c1be0a","Type":"ContainerDied","Data":"02cbffd0c56b32111416a47d48182800fa8c4c3299967c067580b0a051223fe0"}
Sep 11 13:49:17.512140 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.512022    2563 scope.go:117] "RemoveContainer" containerID="4854223a6b47203fe2ec7d247721c933e45916e43d2e9c29c5979be6922a4b5e"
Sep 11 13:49:17.512316 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.512295    2563 scope.go:117] "RemoveContainer" containerID="02cbffd0c56b32111416a47d48182800fa8c4c3299967c067580b0a051223fe0"
Sep 11 13:49:17.512509 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.512481    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wvb64_openshift-console-operator(e274ee6d-0a73-411d-b62a-e41022c1be0a)\"" pod="openshift-console-operator/console-operator-696884fb9c-wvb64" podUID="e274ee6d-0a73-411d-b62a-e41022c1be0a"
Sep 11 13:49:17.608525 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.608479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.608734 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.608654    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:17.608734 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:17.608666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:17.608734 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.608721    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:18.608700363 +0000 UTC m=+54.866900350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:17.608951 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.608839    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:17.608951 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:17.608895    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:18.608877193 +0000 UTC m=+54.867077193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:18.258821 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:18.258789    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2tdnj_78099fbe-6524-4477-8da7-1184017e0ed8/dns-node-resolver/0.log"
Sep 11 13:49:18.519132 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:18.519049    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 13:49:18.519519 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:18.519498    2563 scope.go:117] "RemoveContainer" containerID="02cbffd0c56b32111416a47d48182800fa8c4c3299967c067580b0a051223fe0"
Sep 11 13:49:18.519716 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:18.519696    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wvb64_openshift-console-operator(e274ee6d-0a73-411d-b62a-e41022c1be0a)\"" pod="openshift-console-operator/console-operator-696884fb9c-wvb64" podUID="e274ee6d-0a73-411d-b62a-e41022c1be0a"
Sep 11 13:49:18.619240 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:18.619199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:18.619449 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:18.619282    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:18.619449 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:18.619335    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:20.619319565 +0000 UTC m=+56.877519552 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:18.619449 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:18.619416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:18.619618 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:18.619569    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:18.619735 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:18.619722    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:20.619704238 +0000 UTC m=+56.877904241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:19.334299 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:19.334256    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:19.334556 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:19.334318    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:19.523190 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:19.523156    2563 scope.go:117] "RemoveContainer" containerID="02cbffd0c56b32111416a47d48182800fa8c4c3299967c067580b0a051223fe0"
Sep 11 13:49:19.523651 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:19.523367    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wvb64_openshift-console-operator(e274ee6d-0a73-411d-b62a-e41022c1be0a)\"" pod="openshift-console-operator/console-operator-696884fb9c-wvb64" podUID="e274ee6d-0a73-411d-b62a-e41022c1be0a"
Sep 11 13:49:19.658743 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:19.658715    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nns9f_2adefd72-0d9f-40aa-a193-6b68449ce681/node-ca/0.log"
Sep 11 13:49:20.058818 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:20.058730    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7x4q2_2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3/serve-healthcheck-canary/0.log"
Sep 11 13:49:20.636706 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:20.636665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:20.637121 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:20.636826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:20.637121 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:20.636852    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:20.637121 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:20.636918    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:24.636901214 +0000 UTC m=+60.895101215 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:20.637121 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:20.636944    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:20.637121 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:20.636997    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:24.636983296 +0000 UTC m=+60.895183302 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:24.408088 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:24.408049    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xs2f"
Sep 11 13:49:24.679296 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:24.679200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:24.679480 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:24.679306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:24.679480 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:24.679373    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:24.679480 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:24.679380    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:24.679480 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:24.679425    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:32.679412007 +0000 UTC m=+68.937611993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:24.679480 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:24.679450    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:32.679430622 +0000 UTC m=+68.937630623 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:24.881417 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:24.881376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:24.881631 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:24.881581    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:49:24.881715 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:24.881673    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls podName:541a0cd8-345c-4540-8fb2-7c108a798868 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:40.881650107 +0000 UTC m=+77.139850108 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-lwkxr" (UID: "541a0cd8-345c-4540-8fb2-7c108a798868") : secret "samples-operator-tls" not found
Sep 11 13:49:25.084016 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:25.083920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:25.084016 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:25.083986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:25.084254 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:25.084027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:25.084254 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:25.084090    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:25.084254 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:25.084111    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:49:25.084254 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:25.084170    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls podName:b373479a-c8a5-4ba1-9a2f-918bf37b97cb nodeName:}" failed. No retries permitted until 2026-09-11 13:49:41.084151431 +0000 UTC m=+77.342351434 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hrqh9" (UID: "b373479a-c8a5-4ba1-9a2f-918bf37b97cb") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:49:25.084254 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:25.084199    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:41.084182039 +0000 UTC m=+77.342382040 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : configmap references non-existent config key: service-ca.crt
Sep 11 13:49:25.084254 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:25.084233    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs podName:d967ed16-c979-45a0-aef2-4c4ccf73c12e nodeName:}" failed. No retries permitted until 2026-09-11 13:49:41.084222244 +0000 UTC m=+77.342422232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs") pod "router-default-565b697964-gmnzh" (UID: "d967ed16-c979-45a0-aef2-4c4ccf73c12e") : secret "router-metrics-certs-default" not found
Sep 11 13:49:25.185157 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:25.185119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:25.185331 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:25.185281    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:49:25.185399 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:25.185360    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert podName:4b4a3162-901d-416a-8e12-569888476830 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:41.185340001 +0000 UTC m=+77.443540009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rw8gf" (UID: "4b4a3162-901d-416a-8e12-569888476830") : secret "networking-console-plugin-cert" not found
Sep 11 13:49:30.031676 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.031617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:49:30.034397 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.034368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j72c5\" (UniqueName: \"kubernetes.io/projected/5968c2e1-979b-4278-8e7a-dd70729bf698-kube-api-access-j72c5\") pod \"network-check-target-sfs7q\" (UID: \"5968c2e1-979b-4278-8e7a-dd70729bf698\") " pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:49:30.132488 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.132440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:49:30.132677 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.132613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:49:30.135476 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.135451    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:49:30.135612 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.135542    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:49:30.142787 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:30.142770    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 13:49:30.142857 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:30.142832    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs podName:df547ee7-ed80-4d81-b8e0-b46339dae222 nodeName:}" failed. No retries permitted until 2026-09-11 13:50:34.142812262 +0000 UTC m=+130.401012263 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs") pod "network-metrics-daemon-trgt2" (UID: "df547ee7-ed80-4d81-b8e0-b46339dae222") : secret "metrics-daemon-secret" not found
Sep 11 13:49:30.145166 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.145145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5fa3726c-9b76-4680-8d8b-0931a221c76f-original-pull-secret\") pod \"global-pull-secret-syncer-dg5cq\" (UID: \"5fa3726c-9b76-4680-8d8b-0931a221c76f\") " pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:49:30.147133 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.147108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dg5cq"
Sep 11 13:49:30.157670 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.157648    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlc2t\""
Sep 11 13:49:30.165034 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.164899    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:49:30.234637 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.234614    2563 scope.go:117] "RemoveContainer" containerID="02cbffd0c56b32111416a47d48182800fa8c4c3299967c067580b0a051223fe0"
Sep 11 13:49:30.427376 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.427334    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dg5cq"]
Sep 11 13:49:30.431833 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:30.431803    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa3726c_9b76_4680_8d8b_0931a221c76f.slice/crio-d6d8aa3a435422f58a3b0206eec7348f2fb05b0c2e036f78207810d222ae976d WatchSource:0}: Error finding container d6d8aa3a435422f58a3b0206eec7348f2fb05b0c2e036f78207810d222ae976d: Status 404 returned error can't find the container with id d6d8aa3a435422f58a3b0206eec7348f2fb05b0c2e036f78207810d222ae976d
Sep 11 13:49:30.441304 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.441279    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sfs7q"]
Sep 11 13:49:30.444325 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:30.444305    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5968c2e1_979b_4278_8e7a_dd70729bf698.slice/crio-af8cba2b0ea6dd53725fc0282569d351e48ffd102bbc9974f6c36019cc0fb648 WatchSource:0}: Error finding container af8cba2b0ea6dd53725fc0282569d351e48ffd102bbc9974f6c36019cc0fb648: Status 404 returned error can't find the container with id af8cba2b0ea6dd53725fc0282569d351e48ffd102bbc9974f6c36019cc0fb648
Sep 11 13:49:30.536480 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.536408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:30.536480 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.536460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:30.538791 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.538742    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b97da6d-72f4-4fa8-ae6f-6f8d2f325550-metrics-tls\") pod \"dns-default-hkkx6\" (UID: \"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550\") " pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:30.538920 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.538902    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"image-registry-7d569f65dd-ltwkl\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") " pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:30.558828 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.558788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sfs7q" event={"ID":"5968c2e1-979b-4278-8e7a-dd70729bf698","Type":"ContainerStarted","Data":"ed6a36b79497096efecd6179f26568d779f5fdcf69346c436a92e1a428b31856"}
Sep 11 13:49:30.558967 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.558851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sfs7q" event={"ID":"5968c2e1-979b-4278-8e7a-dd70729bf698","Type":"ContainerStarted","Data":"af8cba2b0ea6dd53725fc0282569d351e48ffd102bbc9974f6c36019cc0fb648"}
Sep 11 13:49:30.558967 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.558911    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:49:30.560874 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.560854    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 13:49:30.560996 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.560948    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wvb64" event={"ID":"e274ee6d-0a73-411d-b62a-e41022c1be0a","Type":"ContainerStarted","Data":"bdeb0eff391fbf2e8fae155d4e0acdc40938795ede716cc2867445d32c100ef9"}
Sep 11 13:49:30.561281 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.561264    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:30.562170 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.562151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dg5cq" event={"ID":"5fa3726c-9b76-4680-8d8b-0931a221c76f","Type":"ContainerStarted","Data":"d6d8aa3a435422f58a3b0206eec7348f2fb05b0c2e036f78207810d222ae976d"}
Sep 11 13:49:30.576546 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.576484    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sfs7q" podStartSLOduration=66.576465284 podStartE2EDuration="1m6.576465284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:30.574551298 +0000 UTC m=+66.832751306" watchObservedRunningTime="2026-09-11 13:49:30.576465284 +0000 UTC m=+66.834665293"
Sep 11 13:49:30.594208 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.594159    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-wvb64" podStartSLOduration=16.27615274 podStartE2EDuration="22.594145914s" totalImagesPullingTime="6.317993174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:30.593793521 +0000 UTC m=+66.851993534" watchObservedRunningTime="2026-09-11 13:49:30.594145914 +0000 UTC m=+66.852345922"
Sep 11 13:49:30.759342 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.759314    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8k7zz\""
Sep 11 13:49:30.767544 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.767518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:30.778771 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.778724    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q92jh\""
Sep 11 13:49:30.786915 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.786824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:30.914894 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:30.914837    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-wvb64"
Sep 11 13:49:31.048105 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:31.048074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d569f65dd-ltwkl"]
Sep 11 13:49:31.050963 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:31.050919    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93ab15a_57e1_443e_9fa0_a6f84b402b9d.slice/crio-e29327c4f293674fd8b5c34d4d1e2b8bd569546b88f0b4500ae03964381b6dd3 WatchSource:0}: Error finding container e29327c4f293674fd8b5c34d4d1e2b8bd569546b88f0b4500ae03964381b6dd3: Status 404 returned error can't find the container with id e29327c4f293674fd8b5c34d4d1e2b8bd569546b88f0b4500ae03964381b6dd3
Sep 11 13:49:31.082351 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:31.082060    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hkkx6"]
Sep 11 13:49:31.090736 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:31.090704    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b97da6d_72f4_4fa8_ae6f_6f8d2f325550.slice/crio-bf209b8aeb670e8185c4eb3a9ed5ba14c65eedb904aaa3297b264699aedcbae2 WatchSource:0}: Error finding container bf209b8aeb670e8185c4eb3a9ed5ba14c65eedb904aaa3297b264699aedcbae2: Status 404 returned error can't find the container with id bf209b8aeb670e8185c4eb3a9ed5ba14c65eedb904aaa3297b264699aedcbae2
Sep 11 13:49:31.568460 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:31.568411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hkkx6" event={"ID":"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550","Type":"ContainerStarted","Data":"bf209b8aeb670e8185c4eb3a9ed5ba14c65eedb904aaa3297b264699aedcbae2"}
Sep 11 13:49:31.575089 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:31.575054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl" event={"ID":"b93ab15a-57e1-443e-9fa0-a6f84b402b9d","Type":"ContainerStarted","Data":"96dc8105d778d6257be74e3aa6d14b24dd576e72b2d0531d97edc88a992a54f5"}
Sep 11 13:49:31.575264 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:31.575116    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl" event={"ID":"b93ab15a-57e1-443e-9fa0-a6f84b402b9d","Type":"ContainerStarted","Data":"e29327c4f293674fd8b5c34d4d1e2b8bd569546b88f0b4500ae03964381b6dd3"}
Sep 11 13:49:31.576000 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:31.575977    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:31.609019 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:31.607548    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl" podStartSLOduration=67.607530654 podStartE2EDuration="1m7.607530654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:31.606979619 +0000 UTC m=+67.865179628" watchObservedRunningTime="2026-09-11 13:49:31.607530654 +0000 UTC m=+67.865730653"
Sep 11 13:49:32.758134 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:32.758096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:32.758554 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:32.758229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:32.758554 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:32.758283    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:32.758554 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:32.758354    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls podName:89f217f3-bc97-424a-9084-27bf60ffcc6a nodeName:}" failed. No retries permitted until 2026-09-11 13:49:48.758338309 +0000 UTC m=+85.016538296 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdn8q" (UID: "89f217f3-bc97-424a-9084-27bf60ffcc6a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:32.758846 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:32.758820    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/89f217f3-bc97-424a-9084-27bf60ffcc6a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:33.585862 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:33.585820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hkkx6" event={"ID":"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550","Type":"ContainerStarted","Data":"465be53159c75cedac29ef75058120b87c4f8a4dbae0dbea448c34453ee1d245"}
Sep 11 13:49:33.586067 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:33.585933    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hkkx6" event={"ID":"4b97da6d-72f4-4fa8-ae6f-6f8d2f325550","Type":"ContainerStarted","Data":"9a49a21681e02f215ff50f313cd5b91e88dc43b7dae5b8a0d48189b2028231c2"}
Sep 11 13:49:33.606199 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:33.606145    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hkkx6" podStartSLOduration=33.991743357 podStartE2EDuration="35.60612664s" totalImagesPullingTime="1.614383283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:33.605336809 +0000 UTC m=+69.863536819" watchObservedRunningTime="2026-09-11 13:49:33.60612664 +0000 UTC m=+69.864326649"
Sep 11 13:49:34.589472 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:34.589435    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:35.594397 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:35.594304    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dg5cq" event={"ID":"5fa3726c-9b76-4680-8d8b-0931a221c76f","Type":"ContainerStarted","Data":"0d474fce333ee2f5bad40c9b4c600c01a7d19c7e51dcc2898635f5eb5920ab94"}
Sep 11 13:49:35.609984 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:35.609929    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dg5cq" podStartSLOduration=66.746177745 podStartE2EDuration="1m11.609912268s" totalImagesPullingTime="4.863734523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:35.608556555 +0000 UTC m=+71.866756564" watchObservedRunningTime="2026-09-11 13:49:35.609912268 +0000 UTC m=+71.868112311"
Sep 11 13:49:40.930621 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:40.930581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:40.933366 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:40.933333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/541a0cd8-345c-4540-8fb2-7c108a798868-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-lwkxr\" (UID: \"541a0cd8-345c-4540-8fb2-7c108a798868\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:40.964609 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:40.964574    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-74rmp"]
Sep 11 13:49:40.967405 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:40.967385    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-74rmp"
Sep 11 13:49:40.971058 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:40.971031    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 13:49:40.971183 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:40.971108    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-sp85c\""
Sep 11 13:49:40.971547 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:40.971528    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 13:49:40.981540 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:40.981515    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-74rmp"]
Sep 11 13:49:41.124279 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.124248    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-46bqg\""
Sep 11 13:49:41.131567 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.131549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"
Sep 11 13:49:41.132974 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.132950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:41.133035 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.132996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:41.133082 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.133044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:41.133160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.133139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v6fv\" (UniqueName: \"kubernetes.io/projected/e819c83e-d5a1-43ba-b752-25ec41500a36-kube-api-access-8v6fv\") pod \"downloads-76f99584f4-74rmp\" (UID: \"e819c83e-d5a1-43ba-b752-25ec41500a36\") " pod="openshift-console/downloads-76f99584f4-74rmp"
Sep 11 13:49:41.133847 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.133827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d967ed16-c979-45a0-aef2-4c4ccf73c12e-service-ca-bundle\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:41.135799 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.135779    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d967ed16-c979-45a0-aef2-4c4ccf73c12e-metrics-certs\") pod \"router-default-565b697964-gmnzh\" (UID: \"d967ed16-c979-45a0-aef2-4c4ccf73c12e\") " pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:41.135996 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.135980    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b373479a-c8a5-4ba1-9a2f-918bf37b97cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hrqh9\" (UID: \"b373479a-c8a5-4ba1-9a2f-918bf37b97cb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:41.234453 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.234411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v6fv\" (UniqueName: \"kubernetes.io/projected/e819c83e-d5a1-43ba-b752-25ec41500a36-kube-api-access-8v6fv\") pod \"downloads-76f99584f4-74rmp\" (UID: \"e819c83e-d5a1-43ba-b752-25ec41500a36\") " pod="openshift-console/downloads-76f99584f4-74rmp"
Sep 11 13:49:41.234629 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.234502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:41.237049 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.237022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4b4a3162-901d-416a-8e12-569888476830-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rw8gf\" (UID: \"4b4a3162-901d-416a-8e12-569888476830\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:41.242351 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.242328    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sqk69\""
Sep 11 13:49:41.248624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.248601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v6fv\" (UniqueName: \"kubernetes.io/projected/e819c83e-d5a1-43ba-b752-25ec41500a36-kube-api-access-8v6fv\") pod \"downloads-76f99584f4-74rmp\" (UID: \"e819c83e-d5a1-43ba-b752-25ec41500a36\") " pod="openshift-console/downloads-76f99584f4-74rmp"
Sep 11 13:49:41.250590 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.250398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"
Sep 11 13:49:41.255291 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.255269    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kwsz9\""
Sep 11 13:49:41.263654 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.263634    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:41.277427 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.277400    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-74rmp"
Sep 11 13:49:41.358152 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.358126    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4lh6n\""
Sep 11 13:49:41.365158 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.365134    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"
Sep 11 13:49:41.404535 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.404125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr"]
Sep 11 13:49:41.526634 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.526610    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9"]
Sep 11 13:49:41.529150 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:41.529116    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb373479a_c8a5_4ba1_9a2f_918bf37b97cb.slice/crio-4c08f5ccebb90eefd3d2b622082c0846e5cad64d79b4f3efe8c4a586681d3d02 WatchSource:0}: Error finding container 4c08f5ccebb90eefd3d2b622082c0846e5cad64d79b4f3efe8c4a586681d3d02: Status 404 returned error can't find the container with id 4c08f5ccebb90eefd3d2b622082c0846e5cad64d79b4f3efe8c4a586681d3d02
Sep 11 13:49:41.553672 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.553645    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-565b697964-gmnzh"]
Sep 11 13:49:41.556399 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:41.556374    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd967ed16_c979_45a0_aef2_4c4ccf73c12e.slice/crio-be4ca12169551738ab8573f4062eb6f489fbc87e1213375dd26b492506901fe1 WatchSource:0}: Error finding container be4ca12169551738ab8573f4062eb6f489fbc87e1213375dd26b492506901fe1: Status 404 returned error can't find the container with id be4ca12169551738ab8573f4062eb6f489fbc87e1213375dd26b492506901fe1
Sep 11 13:49:41.565587 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.565565    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-74rmp"]
Sep 11 13:49:41.568090 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:41.568061    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode819c83e_d5a1_43ba_b752_25ec41500a36.slice/crio-60666e6f8f974a9912a3b7989904950970fcabb0b0fa25706ab02d7881f2ba13 WatchSource:0}: Error finding container 60666e6f8f974a9912a3b7989904950970fcabb0b0fa25706ab02d7881f2ba13: Status 404 returned error can't find the container with id 60666e6f8f974a9912a3b7989904950970fcabb0b0fa25706ab02d7881f2ba13
Sep 11 13:49:41.617787 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.617629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-74rmp" event={"ID":"e819c83e-d5a1-43ba-b752-25ec41500a36","Type":"ContainerStarted","Data":"60666e6f8f974a9912a3b7989904950970fcabb0b0fa25706ab02d7881f2ba13"}
Sep 11 13:49:41.619634 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.619601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-565b697964-gmnzh" event={"ID":"d967ed16-c979-45a0-aef2-4c4ccf73c12e","Type":"ContainerStarted","Data":"be4ca12169551738ab8573f4062eb6f489fbc87e1213375dd26b492506901fe1"}
Sep 11 13:49:41.621268 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.621234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9" event={"ID":"b373479a-c8a5-4ba1-9a2f-918bf37b97cb","Type":"ContainerStarted","Data":"4c08f5ccebb90eefd3d2b622082c0846e5cad64d79b4f3efe8c4a586681d3d02"}
Sep 11 13:49:41.622408 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.622381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr" event={"ID":"541a0cd8-345c-4540-8fb2-7c108a798868","Type":"ContainerStarted","Data":"fb73cf5208da98ad4e7fe87d7c95e3f2681afdce63fa95a2a89929bb96372d33"}
Sep 11 13:49:41.636936 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.636905    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf"]
Sep 11 13:49:41.640112 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:41.639560    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-565b697964-gmnzh" podStartSLOduration=32.639537319 podStartE2EDuration="32.639537319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:41.638151877 +0000 UTC m=+77.896351903" watchObservedRunningTime="2026-09-11 13:49:41.639537319 +0000 UTC m=+77.897737328"
Sep 11 13:49:41.641412 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:41.641383    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4a3162_901d_416a_8e12_569888476830.slice/crio-00055f6764b97b3ddf2e071591051cd078dbcca1f94a5ba9bc5610583d025803 WatchSource:0}: Error finding container 00055f6764b97b3ddf2e071591051cd078dbcca1f94a5ba9bc5610583d025803: Status 404 returned error can't find the container with id 00055f6764b97b3ddf2e071591051cd078dbcca1f94a5ba9bc5610583d025803
Sep 11 13:49:42.264761 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:42.264729    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:42.267342 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:42.267316    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:42.630009 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:42.629950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf" event={"ID":"4b4a3162-901d-416a-8e12-569888476830","Type":"ContainerStarted","Data":"00055f6764b97b3ddf2e071591051cd078dbcca1f94a5ba9bc5610583d025803"}
Sep 11 13:49:42.632988 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:42.632944    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-565b697964-gmnzh" event={"ID":"d967ed16-c979-45a0-aef2-4c4ccf73c12e","Type":"ContainerStarted","Data":"119c863158e00ac10e41ee524ae5884144d371096e661eae8d2898b9fb497006"}
Sep 11 13:49:42.633912 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:42.633884    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:42.635161 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:42.635136    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-565b697964-gmnzh"
Sep 11 13:49:44.597226 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:44.597197    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hkkx6"
Sep 11 13:49:45.647113 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:45.646560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9" event={"ID":"b373479a-c8a5-4ba1-9a2f-918bf37b97cb","Type":"ContainerStarted","Data":"1e47d94afc273bd7e987985b374c351b89c45225284e004034644f19e7bce800"}
Sep 11 13:49:45.648810 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:45.648782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr" event={"ID":"541a0cd8-345c-4540-8fb2-7c108a798868","Type":"ContainerStarted","Data":"03751d17e213c72a6642342f78f6a3a7e5e3dc5255d56d59ee7ac3ea5392c9ea"}
Sep 11 13:49:45.648957 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:45.648830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr" event={"ID":"541a0cd8-345c-4540-8fb2-7c108a798868","Type":"ContainerStarted","Data":"ffa80f02765ee507cb2ccdc9d0582b8e3f872550838887e8ae4dbb1865c27cae"}
Sep 11 13:49:45.650811 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:45.650785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf" event={"ID":"4b4a3162-901d-416a-8e12-569888476830","Type":"ContainerStarted","Data":"5b46c1e49a1a6ed3ce0d311cf985d3371936e7d694f3c164fa17f9cc74494861"}
Sep 11 13:49:45.663498 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:45.663442    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hrqh9" podStartSLOduration=33.365478674 podStartE2EDuration="36.66342228s" totalImagesPullingTime="3.297943606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:45.662814709 +0000 UTC m=+81.921014718" watchObservedRunningTime="2026-09-11 13:49:45.66342228 +0000 UTC m=+81.921622291"
Sep 11 13:49:45.681698 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:45.681638    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-lwkxr" podStartSLOduration=34.300028667 podStartE2EDuration="37.681618106s" totalImagesPullingTime="3.381589439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:45.681020935 +0000 UTC m=+81.939220944" watchObservedRunningTime="2026-09-11 13:49:45.681618106 +0000 UTC m=+81.939818119"
Sep 11 13:49:45.697823 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:45.697766    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rw8gf" podStartSLOduration=33.517188416 podStartE2EDuration="36.697726643s" totalImagesPullingTime="3.180538227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:45.696382734 +0000 UTC m=+81.954582743" watchObservedRunningTime="2026-09-11 13:49:45.697726643 +0000 UTC m=+81.955926653"
Sep 11 13:49:48.807055 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:48.807016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:48.810035 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:48.810003    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/89f217f3-bc97-424a-9084-27bf60ffcc6a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdn8q\" (UID: \"89f217f3-bc97-424a-9084-27bf60ffcc6a\") " pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:49.004667 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:49.004630    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-klbks\""
Sep 11 13:49:49.013217 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:49.013188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xdn8q"
Sep 11 13:49:49.297261 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:49.297228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xdn8q"]
Sep 11 13:49:49.300504 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:49.300471    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f217f3_bc97_424a_9084_27bf60ffcc6a.slice/crio-0c164afbab2085ac602cb6e9ffd1e4059e0fd0a00dc573c4c3b6af91a55a55ca WatchSource:0}: Error finding container 0c164afbab2085ac602cb6e9ffd1e4059e0fd0a00dc573c4c3b6af91a55a55ca: Status 404 returned error can't find the container with id 0c164afbab2085ac602cb6e9ffd1e4059e0fd0a00dc573c4c3b6af91a55a55ca
Sep 11 13:49:49.668364 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:49.668323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xdn8q" event={"ID":"89f217f3-bc97-424a-9084-27bf60ffcc6a","Type":"ContainerStarted","Data":"dfca8f48545559618eb9739c4d6c48fdf614807122800887e929c92625faad1c"}
Sep 11 13:49:49.668564 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:49.668390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xdn8q" event={"ID":"89f217f3-bc97-424a-9084-27bf60ffcc6a","Type":"ContainerStarted","Data":"0c164afbab2085ac602cb6e9ffd1e4059e0fd0a00dc573c4c3b6af91a55a55ca"}
Sep 11 13:49:50.674837 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:50.674796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xdn8q" event={"ID":"89f217f3-bc97-424a-9084-27bf60ffcc6a","Type":"ContainerStarted","Data":"8ae88f5ff066e94a51f99d90712d4637b1ec2e1fb8212c044b2735b012e33e04"}
Sep 11 13:49:50.773820 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:50.773724    2563 patch_prober.go:28] interesting pod/image-registry-7d569f65dd-ltwkl container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 13:49:50.774011 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:50.773852    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl" podUID="b93ab15a-57e1-443e-9fa0-a6f84b402b9d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:49:52.940099 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:52.940063    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r89ms"]
Sep 11 13:49:52.942728 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:52.942702    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:52.945270 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:52.945249    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:49:52.945418 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:52.945307    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gfth2\""
Sep 11 13:49:52.945418 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:52.945322    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:49:52.945528 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:52.945509    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:49:52.945590 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:52.945579    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:49:53.047574 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-wtmp\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.047808 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2n2k\" (UniqueName: \"kubernetes.io/projected/f085f235-e492-4ef6-a75c-e07f6faa8397-kube-api-access-x2n2k\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.047808 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047639    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-tls\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.047808 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-root\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.047808 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f085f235-e492-4ef6-a75c-e07f6faa8397-metrics-client-ca\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.048007 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.048007 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-textfile\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.048007 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-sys\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.048007 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.047918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-accelerators-collector-config\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.148665 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-wtmp\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.148875 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2n2k\" (UniqueName: \"kubernetes.io/projected/f085f235-e492-4ef6-a75c-e07f6faa8397-kube-api-access-x2n2k\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.148875 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-tls\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.148875 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-root\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.148875 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f085f235-e492-4ef6-a75c-e07f6faa8397-metrics-client-ca\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.148875 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.148875 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-wtmp\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.148875 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-textfile\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.149237 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-sys\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.149237 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148921    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-root\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.149237 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.148956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-accelerators-collector-config\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.149237 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:53.149033    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 13:49:53.149237 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:49:53.149092    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-tls podName:f085f235-e492-4ef6-a75c-e07f6faa8397 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:53.649072137 +0000 UTC m=+89.907272129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-tls") pod "node-exporter-r89ms" (UID: "f085f235-e492-4ef6-a75c-e07f6faa8397") : secret "node-exporter-tls" not found
Sep 11 13:49:53.149237 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.149127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f085f235-e492-4ef6-a75c-e07f6faa8397-sys\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.149237 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.149237    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-textfile\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.149736 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.149712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f085f235-e492-4ef6-a75c-e07f6faa8397-metrics-client-ca\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.149848 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.149717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-accelerators-collector-config\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.152027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.152004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.158064 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.158038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2n2k\" (UniqueName: \"kubernetes.io/projected/f085f235-e492-4ef6-a75c-e07f6faa8397-kube-api-access-x2n2k\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.590962 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.590932    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:49:53.654299 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.654262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-tls\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.657380 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.657355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f085f235-e492-4ef6-a75c-e07f6faa8397-node-exporter-tls\") pod \"node-exporter-r89ms\" (UID: \"f085f235-e492-4ef6-a75c-e07f6faa8397\") " pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:53.854011 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:53.853975    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r89ms"
Sep 11 13:49:58.680539 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:49:58.680506    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf085f235_e492_4ef6_a75c_e07f6faa8397.slice/crio-989fa73465c5009689ffbb414f99388f76223ac4df207a0bbb959eb30636b359 WatchSource:0}: Error finding container 989fa73465c5009689ffbb414f99388f76223ac4df207a0bbb959eb30636b359: Status 404 returned error can't find the container with id 989fa73465c5009689ffbb414f99388f76223ac4df207a0bbb959eb30636b359
Sep 11 13:49:58.709173 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:58.709129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r89ms" event={"ID":"f085f235-e492-4ef6-a75c-e07f6faa8397","Type":"ContainerStarted","Data":"989fa73465c5009689ffbb414f99388f76223ac4df207a0bbb959eb30636b359"}
Sep 11 13:49:59.719734 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:59.719693    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xdn8q" event={"ID":"89f217f3-bc97-424a-9084-27bf60ffcc6a","Type":"ContainerStarted","Data":"655e292965ad8545aa55047fe79ecdbda44f6f90f1dfca9c4907c34f743f34b0"}
Sep 11 13:49:59.721490 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:59.721430    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-74rmp" event={"ID":"e819c83e-d5a1-43ba-b752-25ec41500a36","Type":"ContainerStarted","Data":"a9f2621126674764fe17416ad4fb4f0d7a69d330a49d19b9e2f6782dc924cb90"}
Sep 11 13:49:59.721827 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:59.721802    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-74rmp"
Sep 11 13:49:59.732988 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:59.732954    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-74rmp"
Sep 11 13:49:59.738314 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:59.738259    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xdn8q" podStartSLOduration=34.516739735 podStartE2EDuration="43.738238744s" totalImagesPullingTime="9.221499009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:49:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:59.737233086 +0000 UTC m=+95.995433105" watchObservedRunningTime="2026-09-11 13:49:59.738238744 +0000 UTC m=+95.996438760"
Sep 11 13:49:59.754652 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:49:59.754589    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-74rmp" podStartSLOduration=2.598784429 podStartE2EDuration="19.754568557s" totalImagesPullingTime="17.155784128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:59.754058576 +0000 UTC m=+96.012258589" watchObservedRunningTime="2026-09-11 13:49:59.754568557 +0000 UTC m=+96.012768569"
Sep 11 13:50:00.728346 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:00.728304    2563 generic.go:356] "Generic (PLEG): container finished" podID="f085f235-e492-4ef6-a75c-e07f6faa8397" containerID="1e9eb9a211ba0a28dd5e3a8d11f4d810b1ec379f41fbef5551c8e413c751587f" exitCode=0
Sep 11 13:50:00.728865 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:00.728406    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r89ms" event={"ID":"f085f235-e492-4ef6-a75c-e07f6faa8397","Type":"ContainerDied","Data":"1e9eb9a211ba0a28dd5e3a8d11f4d810b1ec379f41fbef5551c8e413c751587f"}
Sep 11 13:50:01.578726 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:01.578692    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sfs7q"
Sep 11 13:50:01.735902 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:01.735788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r89ms" event={"ID":"f085f235-e492-4ef6-a75c-e07f6faa8397","Type":"ContainerStarted","Data":"ff7dd807708052a1d6e8f14ba46ecabc5276fbc7b574eb1f0ef51889cd60d3b9"}
Sep 11 13:50:01.735902 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:01.735863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r89ms" event={"ID":"f085f235-e492-4ef6-a75c-e07f6faa8397","Type":"ContainerStarted","Data":"a820eb7acafede50ed9e89a7e5aeaf3e821927b080e7c6706902ed86d45b40b5"}
Sep 11 13:50:01.755336 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:01.755273    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r89ms" podStartSLOduration=8.696493823 podStartE2EDuration="9.755256103s" totalImagesPullingTime="949.080781ms" totalInitContainerRuntime="109.681499ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:50:01.753397158 +0000 UTC m=+98.011597172" watchObservedRunningTime="2026-09-11 13:50:01.755256103 +0000 UTC m=+98.013456112"
Sep 11 13:50:04.640373 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:04.640333    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d569f65dd-ltwkl"]
Sep 11 13:50:29.664575 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.664513    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl" podUID="b93ab15a-57e1-443e-9fa0-a6f84b402b9d" containerName="registry" containerID="cri-o://96dc8105d778d6257be74e3aa6d14b24dd576e72b2d0531d97edc88a992a54f5" gracePeriod=30
Sep 11 13:50:29.847355 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.847327    2563 generic.go:356] "Generic (PLEG): container finished" podID="b93ab15a-57e1-443e-9fa0-a6f84b402b9d" containerID="96dc8105d778d6257be74e3aa6d14b24dd576e72b2d0531d97edc88a992a54f5" exitCode=0
Sep 11 13:50:29.847522 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.847399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl" event={"ID":"b93ab15a-57e1-443e-9fa0-a6f84b402b9d","Type":"ContainerDied","Data":"96dc8105d778d6257be74e3aa6d14b24dd576e72b2d0531d97edc88a992a54f5"}
Sep 11 13:50:29.936544 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.936473    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:50:29.982117 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982084    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-image-registry-private-configuration\") pod \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") "
Sep 11 13:50:29.982293 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982127    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mw9b\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-kube-api-access-7mw9b\") pod \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") "
Sep 11 13:50:29.982293 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-installation-pull-secrets\") pod \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") "
Sep 11 13:50:29.982293 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982172    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-bound-sa-token\") pod \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") "
Sep 11 13:50:29.982293 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982204    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-certificates\") pod \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") "
Sep 11 13:50:29.982293 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982239    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") pod \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") "
Sep 11 13:50:29.982293 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982292    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-ca-trust-extracted\") pod \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") "
Sep 11 13:50:29.982618 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982329    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-trusted-ca\") pod \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\" (UID: \"b93ab15a-57e1-443e-9fa0-a6f84b402b9d\") "
Sep 11 13:50:29.982735 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.982692    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b93ab15a-57e1-443e-9fa0-a6f84b402b9d" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:29.983165 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.983140    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b93ab15a-57e1-443e-9fa0-a6f84b402b9d" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:29.984990 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.984931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b93ab15a-57e1-443e-9fa0-a6f84b402b9d" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:29.985175 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.985143    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-kube-api-access-7mw9b" (OuterVolumeSpecName: "kube-api-access-7mw9b") pod "b93ab15a-57e1-443e-9fa0-a6f84b402b9d" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d"). InnerVolumeSpecName "kube-api-access-7mw9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:29.985275 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.985183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b93ab15a-57e1-443e-9fa0-a6f84b402b9d" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:29.985351 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.985331    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b93ab15a-57e1-443e-9fa0-a6f84b402b9d" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:29.985933 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.985901    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b93ab15a-57e1-443e-9fa0-a6f84b402b9d" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:29.994006 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:29.993971    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b93ab15a-57e1-443e-9fa0-a6f84b402b9d" (UID: "b93ab15a-57e1-443e-9fa0-a6f84b402b9d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:30.083478 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.083435    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-tls\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:50:30.083478 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.083468    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-ca-trust-extracted\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:50:30.083478 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.083478    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-trusted-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:50:30.083478 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.083486    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-image-registry-private-configuration\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:50:30.083788 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.083495    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mw9b\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-kube-api-access-7mw9b\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:50:30.083788 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.083504    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-installation-pull-secrets\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:50:30.083788 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.083512    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-bound-sa-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:50:30.083788 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.083520    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b93ab15a-57e1-443e-9fa0-a6f84b402b9d-registry-certificates\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:50:30.852526 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.852492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl" event={"ID":"b93ab15a-57e1-443e-9fa0-a6f84b402b9d","Type":"ContainerDied","Data":"e29327c4f293674fd8b5c34d4d1e2b8bd569546b88f0b4500ae03964381b6dd3"}
Sep 11 13:50:30.852971 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.852547    2563 scope.go:117] "RemoveContainer" containerID="96dc8105d778d6257be74e3aa6d14b24dd576e72b2d0531d97edc88a992a54f5"
Sep 11 13:50:30.852971 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.852504    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d569f65dd-ltwkl"
Sep 11 13:50:30.870520 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.870492    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d569f65dd-ltwkl"]
Sep 11 13:50:30.874561 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:30.874534    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d569f65dd-ltwkl"]
Sep 11 13:50:32.238465 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:32.238426    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b93ab15a-57e1-443e-9fa0-a6f84b402b9d" path="/var/lib/kubelet/pods/b93ab15a-57e1-443e-9fa0-a6f84b402b9d/volumes"
Sep 11 13:50:34.215603 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:34.215561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:50:34.218084 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:34.218062    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df547ee7-ed80-4d81-b8e0-b46339dae222-metrics-certs\") pod \"network-metrics-daemon-trgt2\" (UID: \"df547ee7-ed80-4d81-b8e0-b46339dae222\") " pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:50:34.365435 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:34.365406    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6pjch\""
Sep 11 13:50:34.373394 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:34.373372    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trgt2"
Sep 11 13:50:34.674764 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:34.674714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-trgt2"]
Sep 11 13:50:34.678242 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:50:34.678213    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf547ee7_ed80_4d81_b8e0_b46339dae222.slice/crio-97dde78357e0433290979e22e6ea6df9796cbe64d7eb6a310c698b987dd8299b WatchSource:0}: Error finding container 97dde78357e0433290979e22e6ea6df9796cbe64d7eb6a310c698b987dd8299b: Status 404 returned error can't find the container with id 97dde78357e0433290979e22e6ea6df9796cbe64d7eb6a310c698b987dd8299b
Sep 11 13:50:34.868426 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:34.868390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trgt2" event={"ID":"df547ee7-ed80-4d81-b8e0-b46339dae222","Type":"ContainerStarted","Data":"97dde78357e0433290979e22e6ea6df9796cbe64d7eb6a310c698b987dd8299b"}
Sep 11 13:50:36.880435 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:36.880399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trgt2" event={"ID":"df547ee7-ed80-4d81-b8e0-b46339dae222","Type":"ContainerStarted","Data":"98b29e9bbc1f40c19d5b22d2c04fad48aa4b9e4a78096eec1b4d4843bdefc307"}
Sep 11 13:50:36.880895 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:36.880458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trgt2" event={"ID":"df547ee7-ed80-4d81-b8e0-b46339dae222","Type":"ContainerStarted","Data":"1a3b4aca2c77a8220babd4f6dfabaeed6f40e1f9f69fb8500ad628afa24954e2"}
Sep 11 13:50:36.897228 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:36.897176    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-trgt2" podStartSLOduration=131.384810702 podStartE2EDuration="2m12.897160633s" totalImagesPullingTime="1.512349931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:50:36.897061136 +0000 UTC m=+133.155261144" watchObservedRunningTime="2026-09-11 13:50:36.897160633 +0000 UTC m=+133.155360649"
Sep 11 13:50:42.906145 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:42.906109    2563 generic.go:356] "Generic (PLEG): container finished" podID="5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303" containerID="171c7f134b5494e5a3576e5f929cf0a7853b7fdc45a952f5df76997ec6c99146" exitCode=0
Sep 11 13:50:42.906537 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:42.906169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s" event={"ID":"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303","Type":"ContainerDied","Data":"171c7f134b5494e5a3576e5f929cf0a7853b7fdc45a952f5df76997ec6c99146"}
Sep 11 13:50:42.906537 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:42.906476    2563 scope.go:117] "RemoveContainer" containerID="171c7f134b5494e5a3576e5f929cf0a7853b7fdc45a952f5df76997ec6c99146"
Sep 11 13:50:43.911649 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:43.911616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-85r9s" event={"ID":"5641e3ee-4ac9-47ed-ae8c-5cdb7dee2303","Type":"ContainerStarted","Data":"27c2b8aefe64e6a256248d1fc4c4441b2c2933753e04c1e86b465bdc8678879a"}
Sep 11 13:50:46.923920 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:46.923884    2563 generic.go:356] "Generic (PLEG): container finished" podID="59c08c9a-f359-4fa7-b3a2-4b04500dd53e" containerID="249b53de905288c01da8c3d4e6cb10ca30d3a95005faa8a1f23735e5ea2a9f4d" exitCode=0
Sep 11 13:50:46.924405 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:46.923942    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-w7d2b" event={"ID":"59c08c9a-f359-4fa7-b3a2-4b04500dd53e","Type":"ContainerDied","Data":"249b53de905288c01da8c3d4e6cb10ca30d3a95005faa8a1f23735e5ea2a9f4d"}
Sep 11 13:50:46.924405 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:46.924273    2563 scope.go:117] "RemoveContainer" containerID="249b53de905288c01da8c3d4e6cb10ca30d3a95005faa8a1f23735e5ea2a9f4d"
Sep 11 13:50:47.930672 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:47.930634    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-w7d2b" event={"ID":"59c08c9a-f359-4fa7-b3a2-4b04500dd53e","Type":"ContainerStarted","Data":"00e1fd5e98358d3549f32fe3f15350aad0a4a4037959ba28d8c559d8c30e99a8"}
Sep 11 13:50:51.946566 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:51.946478    2563 generic.go:356] "Generic (PLEG): container finished" podID="716a3a57-8906-41de-92c1-b0e0173338ad" containerID="caf0ab3a5df7290ceed7d1687d1b7fea2ae04c90c2302f21046dcb4fcc2f564c" exitCode=0
Sep 11 13:50:51.946566 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:51.946551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76" event={"ID":"716a3a57-8906-41de-92c1-b0e0173338ad","Type":"ContainerDied","Data":"caf0ab3a5df7290ceed7d1687d1b7fea2ae04c90c2302f21046dcb4fcc2f564c"}
Sep 11 13:50:51.946992 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:51.946871    2563 scope.go:117] "RemoveContainer" containerID="caf0ab3a5df7290ceed7d1687d1b7fea2ae04c90c2302f21046dcb4fcc2f564c"
Sep 11 13:50:52.952187 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:50:52.952152    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-tgm76" event={"ID":"716a3a57-8906-41de-92c1-b0e0173338ad","Type":"ContainerStarted","Data":"7dcd427b31c9ac20ff723d792f416c009666c78bcd0851e54c3d29f385dc14f2"}
Sep 11 13:52:09.529236 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.529196    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"]
Sep 11 13:52:09.529683 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.529491    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b93ab15a-57e1-443e-9fa0-a6f84b402b9d" containerName="registry"
Sep 11 13:52:09.529683 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.529502    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b93ab15a-57e1-443e-9fa0-a6f84b402b9d" containerName="registry"
Sep 11 13:52:09.529683 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.529561    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b93ab15a-57e1-443e-9fa0-a6f84b402b9d" containerName="registry"
Sep 11 13:52:09.531575 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.531559    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.534432 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.534408    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:52:09.534581 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.534555    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wcl5z\""
Sep 11 13:52:09.535477 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.535457    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:52:09.546527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.546501    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"]
Sep 11 13:52:09.581663 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.581617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.581870 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.581672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcmz2\" (UniqueName: \"kubernetes.io/projected/3758dda2-0d4d-44c9-a303-0f487107e51e-kube-api-access-pcmz2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.581870 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.581735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.682724 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.682683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.682893 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.682780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.682893 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.682808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcmz2\" (UniqueName: \"kubernetes.io/projected/3758dda2-0d4d-44c9-a303-0f487107e51e-kube-api-access-pcmz2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.683168 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.683145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.683209 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.683179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.691961 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.691941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcmz2\" (UniqueName: \"kubernetes.io/projected/3758dda2-0d4d-44c9-a303-0f487107e51e-kube-api-access-pcmz2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:09.840606 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:09.840516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:10.108569 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:10.108405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"]
Sep 11 13:52:10.111300 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:52:10.111276    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3758dda2_0d4d_44c9_a303_0f487107e51e.slice/crio-2f74527a8fe661b6673e66ab8729d929ba4ed34d21584f4d7465a63ce7855e18 WatchSource:0}: Error finding container 2f74527a8fe661b6673e66ab8729d929ba4ed34d21584f4d7465a63ce7855e18: Status 404 returned error can't find the container with id 2f74527a8fe661b6673e66ab8729d929ba4ed34d21584f4d7465a63ce7855e18
Sep 11 13:52:10.248737 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:10.248695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2" event={"ID":"3758dda2-0d4d-44c9-a303-0f487107e51e","Type":"ContainerStarted","Data":"2f74527a8fe661b6673e66ab8729d929ba4ed34d21584f4d7465a63ce7855e18"}
Sep 11 13:52:16.275071 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:16.275035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2" event={"ID":"3758dda2-0d4d-44c9-a303-0f487107e51e","Type":"ContainerStarted","Data":"66454dbcd28f2832365afde29a72506199cef793d33be885bb53abcd254a5dff"}
Sep 11 13:52:17.279730 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:17.279690    2563 generic.go:356] "Generic (PLEG): container finished" podID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerID="66454dbcd28f2832365afde29a72506199cef793d33be885bb53abcd254a5dff" exitCode=0
Sep 11 13:52:17.280211 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:17.279783    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2" event={"ID":"3758dda2-0d4d-44c9-a303-0f487107e51e","Type":"ContainerDied","Data":"66454dbcd28f2832365afde29a72506199cef793d33be885bb53abcd254a5dff"}
Sep 11 13:52:20.293862 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:20.293823    2563 generic.go:356] "Generic (PLEG): container finished" podID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerID="026709f69710af0e4b22bb52de35545ec0b81746813b84a0e1a83b2b5a9c08af" exitCode=0
Sep 11 13:52:20.294256 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:20.293880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2" event={"ID":"3758dda2-0d4d-44c9-a303-0f487107e51e","Type":"ContainerDied","Data":"026709f69710af0e4b22bb52de35545ec0b81746813b84a0e1a83b2b5a9c08af"}
Sep 11 13:52:27.325817 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:27.325702    2563 generic.go:356] "Generic (PLEG): container finished" podID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerID="3de1af4f8442faa969ccd3efadd768e933316c1f8398c46ce9e654843e803587" exitCode=0
Sep 11 13:52:27.325817 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:27.325787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2" event={"ID":"3758dda2-0d4d-44c9-a303-0f487107e51e","Type":"ContainerDied","Data":"3de1af4f8442faa969ccd3efadd768e933316c1f8398c46ce9e654843e803587"}
Sep 11 13:52:28.456307 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.456280    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:28.552627 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.552573    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-util\") pod \"3758dda2-0d4d-44c9-a303-0f487107e51e\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") "
Sep 11 13:52:28.552849 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.552682    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-bundle\") pod \"3758dda2-0d4d-44c9-a303-0f487107e51e\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") "
Sep 11 13:52:28.552849 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.552714    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcmz2\" (UniqueName: \"kubernetes.io/projected/3758dda2-0d4d-44c9-a303-0f487107e51e-kube-api-access-pcmz2\") pod \"3758dda2-0d4d-44c9-a303-0f487107e51e\" (UID: \"3758dda2-0d4d-44c9-a303-0f487107e51e\") "
Sep 11 13:52:28.553486 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.553455    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-bundle" (OuterVolumeSpecName: "bundle") pod "3758dda2-0d4d-44c9-a303-0f487107e51e" (UID: "3758dda2-0d4d-44c9-a303-0f487107e51e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:28.555218 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.555193    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3758dda2-0d4d-44c9-a303-0f487107e51e-kube-api-access-pcmz2" (OuterVolumeSpecName: "kube-api-access-pcmz2") pod "3758dda2-0d4d-44c9-a303-0f487107e51e" (UID: "3758dda2-0d4d-44c9-a303-0f487107e51e"). InnerVolumeSpecName "kube-api-access-pcmz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:28.557053 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.557032    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-util" (OuterVolumeSpecName: "util") pod "3758dda2-0d4d-44c9-a303-0f487107e51e" (UID: "3758dda2-0d4d-44c9-a303-0f487107e51e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:28.654077 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.654023    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-bundle\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:52:28.654077 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.654071    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcmz2\" (UniqueName: \"kubernetes.io/projected/3758dda2-0d4d-44c9-a303-0f487107e51e-kube-api-access-pcmz2\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:52:28.654077 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:28.654081    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3758dda2-0d4d-44c9-a303-0f487107e51e-util\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:52:29.334676 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:29.334642    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2"
Sep 11 13:52:29.334871 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:29.334646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v7bm2" event={"ID":"3758dda2-0d4d-44c9-a303-0f487107e51e","Type":"ContainerDied","Data":"2f74527a8fe661b6673e66ab8729d929ba4ed34d21584f4d7465a63ce7855e18"}
Sep 11 13:52:29.334871 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:29.334868    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f74527a8fe661b6673e66ab8729d929ba4ed34d21584f4d7465a63ce7855e18"
Sep 11 13:52:32.552353 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.552319    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"]
Sep 11 13:52:32.552861 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.552593    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerName="pull"
Sep 11 13:52:32.552861 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.552603    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerName="pull"
Sep 11 13:52:32.552861 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.552626    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerName="extract"
Sep 11 13:52:32.552861 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.552631    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerName="extract"
Sep 11 13:52:32.552861 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.552639    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerName="util"
Sep 11 13:52:32.552861 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.552644    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerName="util"
Sep 11 13:52:32.552861 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.552682    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3758dda2-0d4d-44c9-a303-0f487107e51e" containerName="extract"
Sep 11 13:52:32.555503 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.555484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"
Sep 11 13:52:32.558422 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.558383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zdhjv\""
Sep 11 13:52:32.558528 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.558453    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:52:32.558528 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.558463    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 11 13:52:32.569188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.569164    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"]
Sep 11 13:52:32.682116 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.682077    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad1e108d-954e-4d67-8628-f3fcafbc28df-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hdj5k\" (UID: \"ad1e108d-954e-4d67-8628-f3fcafbc28df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"
Sep 11 13:52:32.682302 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.682156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwj9m\" (UniqueName: \"kubernetes.io/projected/ad1e108d-954e-4d67-8628-f3fcafbc28df-kube-api-access-nwj9m\") pod \"cert-manager-operator-controller-manager-55c444bf86-hdj5k\" (UID: \"ad1e108d-954e-4d67-8628-f3fcafbc28df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"
Sep 11 13:52:32.783404 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.783371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad1e108d-954e-4d67-8628-f3fcafbc28df-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hdj5k\" (UID: \"ad1e108d-954e-4d67-8628-f3fcafbc28df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"
Sep 11 13:52:32.783541 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.783426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwj9m\" (UniqueName: \"kubernetes.io/projected/ad1e108d-954e-4d67-8628-f3fcafbc28df-kube-api-access-nwj9m\") pod \"cert-manager-operator-controller-manager-55c444bf86-hdj5k\" (UID: \"ad1e108d-954e-4d67-8628-f3fcafbc28df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"
Sep 11 13:52:32.783813 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.783796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad1e108d-954e-4d67-8628-f3fcafbc28df-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hdj5k\" (UID: \"ad1e108d-954e-4d67-8628-f3fcafbc28df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"
Sep 11 13:52:32.792133 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.792107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwj9m\" (UniqueName: \"kubernetes.io/projected/ad1e108d-954e-4d67-8628-f3fcafbc28df-kube-api-access-nwj9m\") pod \"cert-manager-operator-controller-manager-55c444bf86-hdj5k\" (UID: \"ad1e108d-954e-4d67-8628-f3fcafbc28df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"
Sep 11 13:52:32.864529 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:32.864495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"
Sep 11 13:52:33.143939 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:33.143806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k"]
Sep 11 13:52:33.148941 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:52:33.148912    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad1e108d_954e_4d67_8628_f3fcafbc28df.slice/crio-70e8110e1469194b263436d34668596ec113aa9003eb6e45f3e0502f124d7976 WatchSource:0}: Error finding container 70e8110e1469194b263436d34668596ec113aa9003eb6e45f3e0502f124d7976: Status 404 returned error can't find the container with id 70e8110e1469194b263436d34668596ec113aa9003eb6e45f3e0502f124d7976
Sep 11 13:52:33.351689 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:33.351654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k" event={"ID":"ad1e108d-954e-4d67-8628-f3fcafbc28df","Type":"ContainerStarted","Data":"70e8110e1469194b263436d34668596ec113aa9003eb6e45f3e0502f124d7976"}
Sep 11 13:52:35.364408 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:35.364368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k" event={"ID":"ad1e108d-954e-4d67-8628-f3fcafbc28df","Type":"ContainerStarted","Data":"5d57d06a8ad58f10fe15f0e28697238bc849775ba458c521a4c6981781d35eab"}
Sep 11 13:52:35.387322 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:35.387257    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hdj5k" podStartSLOduration=1.7773837829999999 podStartE2EDuration="3.387236361s" totalImagesPullingTime="1.609852578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:52:35.385695469 +0000 UTC m=+251.643895478" watchObservedRunningTime="2026-09-11 13:52:35.387236361 +0000 UTC m=+251.645436370"
Sep 11 13:52:36.475654 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.475619    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"]
Sep 11 13:52:36.479125 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.479103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.481816 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.481794    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:52:36.481958 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.481918    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:52:36.482874 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.482856    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wcl5z\""
Sep 11 13:52:36.493818 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.493792    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"]
Sep 11 13:52:36.509929 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.509899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.510104 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.509938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxjt4\" (UniqueName: \"kubernetes.io/projected/47f221bc-2e63-4db7-a165-492e97fb0be1-kube-api-access-mxjt4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.510104 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.509980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.610340 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.610300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.610340 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.610341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjt4\" (UniqueName: \"kubernetes.io/projected/47f221bc-2e63-4db7-a165-492e97fb0be1-kube-api-access-mxjt4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.610607 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.610368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.610817 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.610794    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.610817 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.610800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.619784 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.619740    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxjt4\" (UniqueName: \"kubernetes.io/projected/47f221bc-2e63-4db7-a165-492e97fb0be1-kube-api-access-mxjt4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.767251 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.767170    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z2z9n"]
Sep 11 13:52:36.770354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.770334    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:36.772917 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.772893    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-nspq6\""
Sep 11 13:52:36.772917 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.772906    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 13:52:36.773099 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.772960    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 13:52:36.781644 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.781620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z2z9n"]
Sep 11 13:52:36.788189 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.788161    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:36.812093 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.812057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5fr\" (UniqueName: \"kubernetes.io/projected/53e3e3de-8c48-4227-8751-1f8eb1998f4f-kube-api-access-sq5fr\") pod \"cert-manager-webhook-6486ffd997-z2z9n\" (UID: \"53e3e3de-8c48-4227-8751-1f8eb1998f4f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:36.812242 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.812184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53e3e3de-8c48-4227-8751-1f8eb1998f4f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z2z9n\" (UID: \"53e3e3de-8c48-4227-8751-1f8eb1998f4f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:36.912785 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.912730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53e3e3de-8c48-4227-8751-1f8eb1998f4f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z2z9n\" (UID: \"53e3e3de-8c48-4227-8751-1f8eb1998f4f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:36.912966 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.912798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5fr\" (UniqueName: \"kubernetes.io/projected/53e3e3de-8c48-4227-8751-1f8eb1998f4f-kube-api-access-sq5fr\") pod \"cert-manager-webhook-6486ffd997-z2z9n\" (UID: \"53e3e3de-8c48-4227-8751-1f8eb1998f4f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:36.924386 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.924350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53e3e3de-8c48-4227-8751-1f8eb1998f4f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z2z9n\" (UID: \"53e3e3de-8c48-4227-8751-1f8eb1998f4f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:36.924461 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:36.924438    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5fr\" (UniqueName: \"kubernetes.io/projected/53e3e3de-8c48-4227-8751-1f8eb1998f4f-kube-api-access-sq5fr\") pod \"cert-manager-webhook-6486ffd997-z2z9n\" (UID: \"53e3e3de-8c48-4227-8751-1f8eb1998f4f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:37.060223 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:37.060127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"]
Sep 11 13:52:37.064724 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:52:37.064694    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f221bc_2e63_4db7_a165_492e97fb0be1.slice/crio-0daa9aa07aedc895a305ef7fc7ca423c07c748e66424b8e654bdd6dc9f2e17e2 WatchSource:0}: Error finding container 0daa9aa07aedc895a305ef7fc7ca423c07c748e66424b8e654bdd6dc9f2e17e2: Status 404 returned error can't find the container with id 0daa9aa07aedc895a305ef7fc7ca423c07c748e66424b8e654bdd6dc9f2e17e2
Sep 11 13:52:37.099555 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:37.099529    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:37.373619 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:37.373585    2563 generic.go:356] "Generic (PLEG): container finished" podID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerID="0bc327f937f95725abbd30ed24f68ca4f753638db4d544d77d57ded267aec0fc" exitCode=0
Sep 11 13:52:37.373869 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:37.373678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts" event={"ID":"47f221bc-2e63-4db7-a165-492e97fb0be1","Type":"ContainerDied","Data":"0bc327f937f95725abbd30ed24f68ca4f753638db4d544d77d57ded267aec0fc"}
Sep 11 13:52:37.373869 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:37.373722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts" event={"ID":"47f221bc-2e63-4db7-a165-492e97fb0be1","Type":"ContainerStarted","Data":"0daa9aa07aedc895a305ef7fc7ca423c07c748e66424b8e654bdd6dc9f2e17e2"}
Sep 11 13:52:37.380192 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:37.380164    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z2z9n"]
Sep 11 13:52:37.382987 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:52:37.382960    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53e3e3de_8c48_4227_8751_1f8eb1998f4f.slice/crio-06888c8b9f6dabcee9355341e0948694954842a988979a19aa401ad7706f9a3e WatchSource:0}: Error finding container 06888c8b9f6dabcee9355341e0948694954842a988979a19aa401ad7706f9a3e: Status 404 returned error can't find the container with id 06888c8b9f6dabcee9355341e0948694954842a988979a19aa401ad7706f9a3e
Sep 11 13:52:38.380897 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:38.380846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n" event={"ID":"53e3e3de-8c48-4227-8751-1f8eb1998f4f","Type":"ContainerStarted","Data":"06888c8b9f6dabcee9355341e0948694954842a988979a19aa401ad7706f9a3e"}
Sep 11 13:52:39.084771 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.084708    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"]
Sep 11 13:52:39.091729 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.091632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"
Sep 11 13:52:39.094510 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.094482    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-kkmdc\""
Sep 11 13:52:39.102838 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.102776    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"]
Sep 11 13:52:39.126806 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.126740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91ed5b25-10d3-4f56-8e49-f147b7d80a40-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-95wg7\" (UID: \"91ed5b25-10d3-4f56-8e49-f147b7d80a40\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"
Sep 11 13:52:39.126991 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.126857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlvzj\" (UniqueName: \"kubernetes.io/projected/91ed5b25-10d3-4f56-8e49-f147b7d80a40-kube-api-access-zlvzj\") pod \"cert-manager-cainjector-559cdd8c87-95wg7\" (UID: \"91ed5b25-10d3-4f56-8e49-f147b7d80a40\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"
Sep 11 13:52:39.227339 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.227237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlvzj\" (UniqueName: \"kubernetes.io/projected/91ed5b25-10d3-4f56-8e49-f147b7d80a40-kube-api-access-zlvzj\") pod \"cert-manager-cainjector-559cdd8c87-95wg7\" (UID: \"91ed5b25-10d3-4f56-8e49-f147b7d80a40\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"
Sep 11 13:52:39.227339 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.227331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91ed5b25-10d3-4f56-8e49-f147b7d80a40-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-95wg7\" (UID: \"91ed5b25-10d3-4f56-8e49-f147b7d80a40\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"
Sep 11 13:52:39.236192 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.236158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91ed5b25-10d3-4f56-8e49-f147b7d80a40-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-95wg7\" (UID: \"91ed5b25-10d3-4f56-8e49-f147b7d80a40\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"
Sep 11 13:52:39.237273 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.237223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlvzj\" (UniqueName: \"kubernetes.io/projected/91ed5b25-10d3-4f56-8e49-f147b7d80a40-kube-api-access-zlvzj\") pod \"cert-manager-cainjector-559cdd8c87-95wg7\" (UID: \"91ed5b25-10d3-4f56-8e49-f147b7d80a40\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"
Sep 11 13:52:39.404984 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:39.404946    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"
Sep 11 13:52:40.915598 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:40.915516    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-95wg7"]
Sep 11 13:52:40.918873 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:52:40.918840    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91ed5b25_10d3_4f56_8e49_f147b7d80a40.slice/crio-f18a05c55f3303dcd34c5556ce0ef31cf36361e4125929f0e890a53aa8ec2715 WatchSource:0}: Error finding container f18a05c55f3303dcd34c5556ce0ef31cf36361e4125929f0e890a53aa8ec2715: Status 404 returned error can't find the container with id f18a05c55f3303dcd34c5556ce0ef31cf36361e4125929f0e890a53aa8ec2715
Sep 11 13:52:41.397946 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:41.397914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n" event={"ID":"53e3e3de-8c48-4227-8751-1f8eb1998f4f","Type":"ContainerStarted","Data":"8ed64b353d1c6e121c307e1d414b4c05207eeec3eb0a9498ec6a976b461c157f"}
Sep 11 13:52:41.398109 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:41.397991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:41.399523 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:41.399488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7" event={"ID":"91ed5b25-10d3-4f56-8e49-f147b7d80a40","Type":"ContainerStarted","Data":"882f792f0e9e72000af2a8f49483ce2ebeebadbd9bc3c3069123201f793400a7"}
Sep 11 13:52:41.399670 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:41.399534    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7" event={"ID":"91ed5b25-10d3-4f56-8e49-f147b7d80a40","Type":"ContainerStarted","Data":"f18a05c55f3303dcd34c5556ce0ef31cf36361e4125929f0e890a53aa8ec2715"}
Sep 11 13:52:41.401468 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:41.401437    2563 generic.go:356] "Generic (PLEG): container finished" podID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerID="27294d7ca35153b452b730edc713fd54d8e4ecb4aa34f2f35e2c7f2423e4db9a" exitCode=0
Sep 11 13:52:41.401582 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:41.401498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts" event={"ID":"47f221bc-2e63-4db7-a165-492e97fb0be1","Type":"ContainerDied","Data":"27294d7ca35153b452b730edc713fd54d8e4ecb4aa34f2f35e2c7f2423e4db9a"}
Sep 11 13:52:41.417009 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:41.416962    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n" podStartSLOduration=2.164993462 podStartE2EDuration="5.416946556s" totalImagesPullingTime="3.251953094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:52:41.416078373 +0000 UTC m=+257.674278382" watchObservedRunningTime="2026-09-11 13:52:41.416946556 +0000 UTC m=+257.675146566"
Sep 11 13:52:41.449237 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:41.449179    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-95wg7" podStartSLOduration=2.449156411 podStartE2EDuration="2.449156411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:52:41.448445061 +0000 UTC m=+257.706645081" watchObservedRunningTime="2026-09-11 13:52:41.449156411 +0000 UTC m=+257.707356421"
Sep 11 13:52:42.408785 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:42.408732    2563 generic.go:356] "Generic (PLEG): container finished" podID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerID="acb37763ec0ed981d002811483e585b5d93a326c81e5a2d8c8dc8621ded6a80f" exitCode=0
Sep 11 13:52:42.409208 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:42.408786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts" event={"ID":"47f221bc-2e63-4db7-a165-492e97fb0be1","Type":"ContainerDied","Data":"acb37763ec0ed981d002811483e585b5d93a326c81e5a2d8c8dc8621ded6a80f"}
Sep 11 13:52:43.540530 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.540501    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:43.559110 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.558795    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-bundle\") pod \"47f221bc-2e63-4db7-a165-492e97fb0be1\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") "
Sep 11 13:52:43.559110 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.558844    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-util\") pod \"47f221bc-2e63-4db7-a165-492e97fb0be1\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") "
Sep 11 13:52:43.559110 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.558926    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxjt4\" (UniqueName: \"kubernetes.io/projected/47f221bc-2e63-4db7-a165-492e97fb0be1-kube-api-access-mxjt4\") pod \"47f221bc-2e63-4db7-a165-492e97fb0be1\" (UID: \"47f221bc-2e63-4db7-a165-492e97fb0be1\") "
Sep 11 13:52:43.559482 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.559450    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-bundle" (OuterVolumeSpecName: "bundle") pod "47f221bc-2e63-4db7-a165-492e97fb0be1" (UID: "47f221bc-2e63-4db7-a165-492e97fb0be1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:43.561915 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.561879    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47f221bc-2e63-4db7-a165-492e97fb0be1-kube-api-access-mxjt4" (OuterVolumeSpecName: "kube-api-access-mxjt4") pod "47f221bc-2e63-4db7-a165-492e97fb0be1" (UID: "47f221bc-2e63-4db7-a165-492e97fb0be1"). InnerVolumeSpecName "kube-api-access-mxjt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:43.565728 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.565700    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-util" (OuterVolumeSpecName: "util") pod "47f221bc-2e63-4db7-a165-492e97fb0be1" (UID: "47f221bc-2e63-4db7-a165-492e97fb0be1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:43.659723 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.659677    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-bundle\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:52:43.659723 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.659711    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f221bc-2e63-4db7-a165-492e97fb0be1-util\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:52:43.659723 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:43.659725    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxjt4\" (UniqueName: \"kubernetes.io/projected/47f221bc-2e63-4db7-a165-492e97fb0be1-kube-api-access-mxjt4\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:52:44.420596 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:44.420562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts" event={"ID":"47f221bc-2e63-4db7-a165-492e97fb0be1","Type":"ContainerDied","Data":"0daa9aa07aedc895a305ef7fc7ca423c07c748e66424b8e654bdd6dc9f2e17e2"}
Sep 11 13:52:44.420782 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:44.420617    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0daa9aa07aedc895a305ef7fc7ca423c07c748e66424b8e654bdd6dc9f2e17e2"
Sep 11 13:52:44.420782 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:44.420577    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9prts"
Sep 11 13:52:47.411897 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:47.411820    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z2z9n"
Sep 11 13:52:59.390957 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.390919    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"]
Sep 11 13:52:59.391437 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.391198    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerName="pull"
Sep 11 13:52:59.391437 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.391208    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerName="pull"
Sep 11 13:52:59.391437 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.391216    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerName="extract"
Sep 11 13:52:59.391437 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.391221    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerName="extract"
Sep 11 13:52:59.391437 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.391233    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerName="util"
Sep 11 13:52:59.391437 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.391237    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerName="util"
Sep 11 13:52:59.391437 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.391293    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="47f221bc-2e63-4db7-a165-492e97fb0be1" containerName="extract"
Sep 11 13:52:59.431852 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.431811    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"]
Sep 11 13:52:59.432048 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.431957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.434782 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.434742    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:52:59.434909 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.434742    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:52:59.435782 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.435746    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wcl5z\""
Sep 11 13:52:59.482276 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.482238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.482460 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.482297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj55x\" (UniqueName: \"kubernetes.io/projected/7a3cf504-a264-4297-99a7-16f2694b29d6-kube-api-access-tj55x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.482460 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.482391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.583391 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.583355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.583661 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.583420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj55x\" (UniqueName: \"kubernetes.io/projected/7a3cf504-a264-4297-99a7-16f2694b29d6-kube-api-access-tj55x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.583661 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.583463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.583836 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.583738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.583836 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.583809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.592162 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.592138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj55x\" (UniqueName: \"kubernetes.io/projected/7a3cf504-a264-4297-99a7-16f2694b29d6-kube-api-access-tj55x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:52:59.741785 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:52:59.741676    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:53:00.030972 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:00.030882    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"]
Sep 11 13:53:00.033706 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:53:00.033674    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3cf504_a264_4297_99a7_16f2694b29d6.slice/crio-1407f5c7c06d7dc0669ed5ff8c4f7f6c763ad6d862307e9bc12eb201e88d22af WatchSource:0}: Error finding container 1407f5c7c06d7dc0669ed5ff8c4f7f6c763ad6d862307e9bc12eb201e88d22af: Status 404 returned error can't find the container with id 1407f5c7c06d7dc0669ed5ff8c4f7f6c763ad6d862307e9bc12eb201e88d22af
Sep 11 13:53:00.489380 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:00.489348    2563 generic.go:356] "Generic (PLEG): container finished" podID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerID="cd6b152140b5358138d5166c08a8ce35f03fc8ca3113c2812828adf6e34b7937" exitCode=0
Sep 11 13:53:00.489867 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:00.489434    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq" event={"ID":"7a3cf504-a264-4297-99a7-16f2694b29d6","Type":"ContainerDied","Data":"cd6b152140b5358138d5166c08a8ce35f03fc8ca3113c2812828adf6e34b7937"}
Sep 11 13:53:00.489867 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:00.489486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq" event={"ID":"7a3cf504-a264-4297-99a7-16f2694b29d6","Type":"ContainerStarted","Data":"1407f5c7c06d7dc0669ed5ff8c4f7f6c763ad6d862307e9bc12eb201e88d22af"}
Sep 11 13:53:02.500252 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:02.500214    2563 generic.go:356] "Generic (PLEG): container finished" podID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerID="197d51abae37f9bd348be312bdd240b1132570502e8e47aeee3bc9a47ab53b9c" exitCode=0
Sep 11 13:53:02.500708 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:02.500301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq" event={"ID":"7a3cf504-a264-4297-99a7-16f2694b29d6","Type":"ContainerDied","Data":"197d51abae37f9bd348be312bdd240b1132570502e8e47aeee3bc9a47ab53b9c"}
Sep 11 13:53:03.507540 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:03.507504    2563 generic.go:356] "Generic (PLEG): container finished" podID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerID="d106e36f2fec47e98aa45b1487c9410a52bc8db68ce31dce4f7f013ca5491324" exitCode=0
Sep 11 13:53:03.507989 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:03.507586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq" event={"ID":"7a3cf504-a264-4297-99a7-16f2694b29d6","Type":"ContainerDied","Data":"d106e36f2fec47e98aa45b1487c9410a52bc8db68ce31dce4f7f013ca5491324"}
Sep 11 13:53:04.644945 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.644911    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:53:04.723851 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.723812    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-bundle\") pod \"7a3cf504-a264-4297-99a7-16f2694b29d6\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") "
Sep 11 13:53:04.724049 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.723868    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj55x\" (UniqueName: \"kubernetes.io/projected/7a3cf504-a264-4297-99a7-16f2694b29d6-kube-api-access-tj55x\") pod \"7a3cf504-a264-4297-99a7-16f2694b29d6\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") "
Sep 11 13:53:04.724049 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.723898    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-util\") pod \"7a3cf504-a264-4297-99a7-16f2694b29d6\" (UID: \"7a3cf504-a264-4297-99a7-16f2694b29d6\") "
Sep 11 13:53:04.724607 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.724580    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-bundle" (OuterVolumeSpecName: "bundle") pod "7a3cf504-a264-4297-99a7-16f2694b29d6" (UID: "7a3cf504-a264-4297-99a7-16f2694b29d6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:04.726401 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.726377    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a3cf504-a264-4297-99a7-16f2694b29d6-kube-api-access-tj55x" (OuterVolumeSpecName: "kube-api-access-tj55x") pod "7a3cf504-a264-4297-99a7-16f2694b29d6" (UID: "7a3cf504-a264-4297-99a7-16f2694b29d6"). InnerVolumeSpecName "kube-api-access-tj55x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:04.729468 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.729425    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-util" (OuterVolumeSpecName: "util") pod "7a3cf504-a264-4297-99a7-16f2694b29d6" (UID: "7a3cf504-a264-4297-99a7-16f2694b29d6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:04.825141 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.825039    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-bundle\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:04.825141 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.825076    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj55x\" (UniqueName: \"kubernetes.io/projected/7a3cf504-a264-4297-99a7-16f2694b29d6-kube-api-access-tj55x\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:04.825141 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:04.825090    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a3cf504-a264-4297-99a7-16f2694b29d6-util\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:05.518958 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:05.518928    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq"
Sep 11 13:53:05.518958 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:05.518940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkhqlq" event={"ID":"7a3cf504-a264-4297-99a7-16f2694b29d6","Type":"ContainerDied","Data":"1407f5c7c06d7dc0669ed5ff8c4f7f6c763ad6d862307e9bc12eb201e88d22af"}
Sep 11 13:53:05.519263 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:05.518990    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1407f5c7c06d7dc0669ed5ff8c4f7f6c763ad6d862307e9bc12eb201e88d22af"
Sep 11 13:53:16.125257 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.125213    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"]
Sep 11 13:53:16.125874 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.125653    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerName="extract"
Sep 11 13:53:16.125874 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.125671    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerName="extract"
Sep 11 13:53:16.125874 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.125690    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerName="pull"
Sep 11 13:53:16.125874 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.125698    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerName="pull"
Sep 11 13:53:16.125874 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.125717    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerName="util"
Sep 11 13:53:16.125874 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.125726    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerName="util"
Sep 11 13:53:16.126173 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.125877    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3cf504-a264-4297-99a7-16f2694b29d6" containerName="extract"
Sep 11 13:53:16.127736 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.127714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.132357 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.132335    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 13:53:16.132778 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.132736    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 13:53:16.132834 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.132793    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 13:53:16.134074 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.134052    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:53:16.134199 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.134054    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ql4wz\""
Sep 11 13:53:16.136448 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.136407    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 13:53:16.147830 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.147799    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"]
Sep 11 13:53:16.226479 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.226439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/624e3791-d936-477d-b457-d1bd485a16d1-manager-config\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.226665 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.226510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhwgs\" (UniqueName: \"kubernetes.io/projected/624e3791-d936-477d-b457-d1bd485a16d1-kube-api-access-nhwgs\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.226665 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.226534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/624e3791-d936-477d-b457-d1bd485a16d1-metrics-cert\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.226665 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.226557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/624e3791-d936-477d-b457-d1bd485a16d1-cert\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.327502 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.327461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhwgs\" (UniqueName: \"kubernetes.io/projected/624e3791-d936-477d-b457-d1bd485a16d1-kube-api-access-nhwgs\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.327502 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.327503    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/624e3791-d936-477d-b457-d1bd485a16d1-metrics-cert\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.327714 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.327531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/624e3791-d936-477d-b457-d1bd485a16d1-cert\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.327714 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.327575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/624e3791-d936-477d-b457-d1bd485a16d1-manager-config\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.328312 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.328230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/624e3791-d936-477d-b457-d1bd485a16d1-manager-config\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.330177 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.330155    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/624e3791-d936-477d-b457-d1bd485a16d1-metrics-cert\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.330283 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.330271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/624e3791-d936-477d-b457-d1bd485a16d1-cert\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.339152 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.339114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhwgs\" (UniqueName: \"kubernetes.io/projected/624e3791-d936-477d-b457-d1bd485a16d1-kube-api-access-nhwgs\") pod \"lws-controller-manager-8546dd6968-rvcwb\" (UID: \"624e3791-d936-477d-b457-d1bd485a16d1\") " pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.437993 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.437893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:16.504578 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.504530    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"]
Sep 11 13:53:16.511052 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.511022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.515587 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.515135    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:53:16.515587 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.515295    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8wssf\""
Sep 11 13:53:16.515587 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.515516    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:53:16.515954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.515628    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:53:16.515954 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.515667    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:53:16.536107 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.536074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"]
Sep 11 13:53:16.629681 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.629629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d765dc6a-5737-4b56-9f63-7f7c51efd209-manifests\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.629886 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.629695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d765dc6a-5737-4b56-9f63-7f7c51efd209-webhook-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.629886 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.629811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d765dc6a-5737-4b56-9f63-7f7c51efd209-tmp\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.629886 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.629845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d765dc6a-5737-4b56-9f63-7f7c51efd209-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.629886 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.629864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47t6w\" (UniqueName: \"kubernetes.io/projected/d765dc6a-5737-4b56-9f63-7f7c51efd209-kube-api-access-47t6w\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.721810 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.721711    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"]
Sep 11 13:53:16.730987 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.730956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d765dc6a-5737-4b56-9f63-7f7c51efd209-webhook-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.731107 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.731016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d765dc6a-5737-4b56-9f63-7f7c51efd209-tmp\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.731107 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.731040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d765dc6a-5737-4b56-9f63-7f7c51efd209-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.731107 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.731057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47t6w\" (UniqueName: \"kubernetes.io/projected/d765dc6a-5737-4b56-9f63-7f7c51efd209-kube-api-access-47t6w\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.731107 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.731090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d765dc6a-5737-4b56-9f63-7f7c51efd209-manifests\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.731483 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.731455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d765dc6a-5737-4b56-9f63-7f7c51efd209-manifests\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.733600 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.733579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d765dc6a-5737-4b56-9f63-7f7c51efd209-tmp\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.733723 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.733705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d765dc6a-5737-4b56-9f63-7f7c51efd209-webhook-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.733887 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.733870    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d765dc6a-5737-4b56-9f63-7f7c51efd209-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.740467 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.740436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47t6w\" (UniqueName: \"kubernetes.io/projected/d765dc6a-5737-4b56-9f63-7f7c51efd209-kube-api-access-47t6w\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2bc9x\" (UID: \"d765dc6a-5737-4b56-9f63-7f7c51efd209\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.773382 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.773336    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"]
Sep 11 13:53:16.776555 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.776537    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.779301 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.779276    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:53:16.779301 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.779300    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wcl5z\""
Sep 11 13:53:16.779561 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.779308    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:53:16.787087 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.787061    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"]
Sep 11 13:53:16.824123 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.824086    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:16.831666 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.831636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.831789 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.831710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.831789 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.831730    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ncrb\" (UniqueName: \"kubernetes.io/projected/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-kube-api-access-8ncrb\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.932345 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.932307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.932559 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.932356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ncrb\" (UniqueName: \"kubernetes.io/projected/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-kube-api-access-8ncrb\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.932559 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.932397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.932733 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.932709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.932828 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.932776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:16.942200 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:16.942170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ncrb\" (UniqueName: \"kubernetes.io/projected/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-kube-api-access-8ncrb\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:17.086919 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:17.086820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:17.118686 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:17.118636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"]
Sep 11 13:53:17.121890 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:53:17.121853    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd765dc6a_5737_4b56_9f63_7f7c51efd209.slice/crio-c5e799a73b2e4e3af4fbf1cdc74019af1574526df752d976f4254992aa88af84 WatchSource:0}: Error finding container c5e799a73b2e4e3af4fbf1cdc74019af1574526df752d976f4254992aa88af84: Status 404 returned error can't find the container with id c5e799a73b2e4e3af4fbf1cdc74019af1574526df752d976f4254992aa88af84
Sep 11 13:53:17.371852 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:17.371828    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"]
Sep 11 13:53:17.374576 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:53:17.374538    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee1ae17_ea6d_4743_a3e0_45687f8a02f8.slice/crio-6199892b14ddbfec9d0bdf06354ae09a9e67cab7f14395b4243a59cb08a610bd WatchSource:0}: Error finding container 6199892b14ddbfec9d0bdf06354ae09a9e67cab7f14395b4243a59cb08a610bd: Status 404 returned error can't find the container with id 6199892b14ddbfec9d0bdf06354ae09a9e67cab7f14395b4243a59cb08a610bd
Sep 11 13:53:17.579020 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:17.578980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x" event={"ID":"d765dc6a-5737-4b56-9f63-7f7c51efd209","Type":"ContainerStarted","Data":"c5e799a73b2e4e3af4fbf1cdc74019af1574526df752d976f4254992aa88af84"}
Sep 11 13:53:17.580224 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:17.580188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb" event={"ID":"624e3791-d936-477d-b457-d1bd485a16d1","Type":"ContainerStarted","Data":"6a158f4f72abde8a7ac0349d920c36bae59e0f47c1f164e4de27e2af322ad5cc"}
Sep 11 13:53:17.581718 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:17.581688    2563 generic.go:356] "Generic (PLEG): container finished" podID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerID="a27a0977086636fa5b21996fbc726630f8ab563a44f80432dac06ee349d3787a" exitCode=0
Sep 11 13:53:17.581884 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:17.581723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq" event={"ID":"bee1ae17-ea6d-4743-a3e0-45687f8a02f8","Type":"ContainerDied","Data":"a27a0977086636fa5b21996fbc726630f8ab563a44f80432dac06ee349d3787a"}
Sep 11 13:53:17.581884 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:17.581770    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq" event={"ID":"bee1ae17-ea6d-4743-a3e0-45687f8a02f8","Type":"ContainerStarted","Data":"6199892b14ddbfec9d0bdf06354ae09a9e67cab7f14395b4243a59cb08a610bd"}
Sep 11 13:53:18.591560 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:18.591470    2563 generic.go:356] "Generic (PLEG): container finished" podID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerID="31482004e965093bd82d6d2606e2052c4c383c85f1ff62da696e99f755fa7381" exitCode=0
Sep 11 13:53:18.592062 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:18.591620    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq" event={"ID":"bee1ae17-ea6d-4743-a3e0-45687f8a02f8","Type":"ContainerDied","Data":"31482004e965093bd82d6d2606e2052c4c383c85f1ff62da696e99f755fa7381"}
Sep 11 13:53:21.610410 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:21.610372    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb" event={"ID":"624e3791-d936-477d-b457-d1bd485a16d1","Type":"ContainerStarted","Data":"d8c2666cc2fd2d2191bc65609057a36b02d4578294fea616f94650f5553809ac"}
Sep 11 13:53:21.610919 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:21.610437    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:21.612624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:21.612600    2563 generic.go:356] "Generic (PLEG): container finished" podID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerID="71de65f7f464406c4ee6a48613d1a4eb8c9ff9257c57a3c5d3c5b3891eca77fd" exitCode=0
Sep 11 13:53:21.612743 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:21.612671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq" event={"ID":"bee1ae17-ea6d-4743-a3e0-45687f8a02f8","Type":"ContainerDied","Data":"71de65f7f464406c4ee6a48613d1a4eb8c9ff9257c57a3c5d3c5b3891eca77fd"}
Sep 11 13:53:21.614257 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:21.614234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x" event={"ID":"d765dc6a-5737-4b56-9f63-7f7c51efd209","Type":"ContainerStarted","Data":"2bdbd5904a5ce8250bcca69e007c81eb1291b4a30375d637cbc788bbac7fd596"}
Sep 11 13:53:21.629890 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:21.629515    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb" podStartSLOduration=1.7758190809999999 podStartE2EDuration="5.629497659s" totalImagesPullingTime="3.853678578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:21.628495868 +0000 UTC m=+297.886695877" watchObservedRunningTime="2026-09-11 13:53:21.629497659 +0000 UTC m=+297.887697670"
Sep 11 13:53:22.620370 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.620332    2563 generic.go:356] "Generic (PLEG): container finished" podID="d765dc6a-5737-4b56-9f63-7f7c51efd209" containerID="2bdbd5904a5ce8250bcca69e007c81eb1291b4a30375d637cbc788bbac7fd596" exitCode=0
Sep 11 13:53:22.620777 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.620375    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x" event={"ID":"d765dc6a-5737-4b56-9f63-7f7c51efd209","Type":"ContainerDied","Data":"2bdbd5904a5ce8250bcca69e007c81eb1291b4a30375d637cbc788bbac7fd596"}
Sep 11 13:53:22.748404 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.748376    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:22.890397 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.890295    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ncrb\" (UniqueName: \"kubernetes.io/projected/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-kube-api-access-8ncrb\") pod \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") "
Sep 11 13:53:22.890397 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.890379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-util\") pod \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") "
Sep 11 13:53:22.890602 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.890450    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-bundle\") pod \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\" (UID: \"bee1ae17-ea6d-4743-a3e0-45687f8a02f8\") "
Sep 11 13:53:22.891191 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.891157    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-bundle" (OuterVolumeSpecName: "bundle") pod "bee1ae17-ea6d-4743-a3e0-45687f8a02f8" (UID: "bee1ae17-ea6d-4743-a3e0-45687f8a02f8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:22.892828 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.892802    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-kube-api-access-8ncrb" (OuterVolumeSpecName: "kube-api-access-8ncrb") pod "bee1ae17-ea6d-4743-a3e0-45687f8a02f8" (UID: "bee1ae17-ea6d-4743-a3e0-45687f8a02f8"). InnerVolumeSpecName "kube-api-access-8ncrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:22.971507 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.971443    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-util" (OuterVolumeSpecName: "util") pod "bee1ae17-ea6d-4743-a3e0-45687f8a02f8" (UID: "bee1ae17-ea6d-4743-a3e0-45687f8a02f8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:22.991273 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.991237    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-bundle\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:22.991273 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.991271    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ncrb\" (UniqueName: \"kubernetes.io/projected/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-kube-api-access-8ncrb\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:22.991422 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:22.991282    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bee1ae17-ea6d-4743-a3e0-45687f8a02f8-util\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:23.627502 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:23.627472    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq"
Sep 11 13:53:23.627970 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:23.627468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b087zfq" event={"ID":"bee1ae17-ea6d-4743-a3e0-45687f8a02f8","Type":"ContainerDied","Data":"6199892b14ddbfec9d0bdf06354ae09a9e67cab7f14395b4243a59cb08a610bd"}
Sep 11 13:53:23.627970 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:23.627616    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6199892b14ddbfec9d0bdf06354ae09a9e67cab7f14395b4243a59cb08a610bd"
Sep 11 13:53:23.629619 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:23.629596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x" event={"ID":"d765dc6a-5737-4b56-9f63-7f7c51efd209","Type":"ContainerStarted","Data":"b98dd21e7ce0e6ef5655c4f950f3e566014b6db899232376bbdaba5d89b677f8"}
Sep 11 13:53:23.629811 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:23.629700    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:23.653294 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:23.653236    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x" podStartSLOduration=2.658021916 podStartE2EDuration="7.653211939s" totalImagesPullingTime="3.96626672s" totalInitContainerRuntime="1.028923303s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:23.652568368 +0000 UTC m=+299.910768378" watchObservedRunningTime="2026-09-11 13:53:23.653211939 +0000 UTC m=+299.911411950"
Sep 11 13:53:24.181988 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:24.181956    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 13:53:24.192966 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:24.192936    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 13:53:24.203527 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:24.203504    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:53:32.623199 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:32.623163    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8546dd6968-rvcwb"
Sep 11 13:53:34.637151 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:34.637122    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2bc9x"
Sep 11 13:53:45.611229 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.611186    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"]
Sep 11 13:53:45.611624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.611494    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerName="extract"
Sep 11 13:53:45.611624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.611505    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerName="extract"
Sep 11 13:53:45.611624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.611516    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerName="util"
Sep 11 13:53:45.611624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.611522    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerName="util"
Sep 11 13:53:45.611624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.611552    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerName="pull"
Sep 11 13:53:45.611624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.611558    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerName="pull"
Sep 11 13:53:45.611624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.611613    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bee1ae17-ea6d-4743-a3e0-45687f8a02f8" containerName="extract"
Sep 11 13:53:45.613363 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.613348    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.616936 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.616905    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kqh22\""
Sep 11 13:53:45.616936 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.616905    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 13:53:45.628932 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.628906    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"]
Sep 11 13:53:45.675804 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.675735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.675804 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.675799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83316f5f-166d-4495-8901-129fdffa0ad9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.676038 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.675820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgzcf\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-kube-api-access-dgzcf\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.676038 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.675836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.676038 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.675864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.676038 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.675914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.676038 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.675946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.676038 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.675960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83316f5f-166d-4495-8901-129fdffa0ad9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.676038 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.676009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777079 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777079 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83316f5f-166d-4495-8901-129fdffa0ad9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgzcf\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-kube-api-access-dgzcf\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83316f5f-166d-4495-8901-129fdffa0ad9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777354 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777745 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.777745 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.777667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.778030 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.778005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.778138 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.778115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.778182 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.778142    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83316f5f-166d-4495-8901-129fdffa0ad9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.779728 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.779707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.779852 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.779804    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83316f5f-166d-4495-8901-129fdffa0ad9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.789158 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.789126    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.789476 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.789453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgzcf\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-kube-api-access-dgzcf\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.878384 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.878294    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"]
Sep 11 13:53:45.881456 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.881439    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.893474 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.893446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"]
Sep 11 13:53:45.924385 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.924340    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:45.979360 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.979511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.979511 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.979597 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.979597 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979563    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.979597 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pxtk\" (UniqueName: \"kubernetes.io/projected/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-kube-api-access-2pxtk\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.979704 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.979704 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:45.979704 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:45.979671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.080798 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.080730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.080990 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.080831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.080990 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.080861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pxtk\" (UniqueName: \"kubernetes.io/projected/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-kube-api-access-2pxtk\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081074 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081116 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081202 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081202 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081279 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081333 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081388 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081388 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081584 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081732 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.081885 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.081836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.083634 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.083610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.083880 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.083864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.089552 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.089523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pxtk\" (UniqueName: \"kubernetes.io/projected/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-kube-api-access-2pxtk\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.089784 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.089768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b6449a4-e6ad-4bf8-b322-c6b995583ee8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm\" (UID: \"2b6449a4-e6ad-4bf8-b322-c6b995583ee8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.192368 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.192286    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:46.202198 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.202172    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"]
Sep 11 13:53:46.203548 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:53:46.203518    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83316f5f_166d_4495_8901_129fdffa0ad9.slice/crio-c91d9ea5491bec4666dca505f94a24ef9e9e6d3dc7768a5807ad409be402cbe1 WatchSource:0}: Error finding container c91d9ea5491bec4666dca505f94a24ef9e9e6d3dc7768a5807ad409be402cbe1: Status 404 returned error can't find the container with id c91d9ea5491bec4666dca505f94a24ef9e9e6d3dc7768a5807ad409be402cbe1
Sep 11 13:53:46.205728 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.205711    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:53:46.469921 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.469896    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"]
Sep 11 13:53:46.471913 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:53:46.471889    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6449a4_e6ad_4bf8_b322_c6b995583ee8.slice/crio-dd9369014ed4f181e66a6a5bd261eff259df1fbcf198dc4268ed268f6b3abede WatchSource:0}: Error finding container dd9369014ed4f181e66a6a5bd261eff259df1fbcf198dc4268ed268f6b3abede: Status 404 returned error can't find the container with id dd9369014ed4f181e66a6a5bd261eff259df1fbcf198dc4268ed268f6b3abede
Sep 11 13:53:46.736356 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.736258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6" event={"ID":"83316f5f-166d-4495-8901-129fdffa0ad9","Type":"ContainerStarted","Data":"c91d9ea5491bec4666dca505f94a24ef9e9e6d3dc7768a5807ad409be402cbe1"}
Sep 11 13:53:46.737623 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:46.737598    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm" event={"ID":"2b6449a4-e6ad-4bf8-b322-c6b995583ee8","Type":"ContainerStarted","Data":"dd9369014ed4f181e66a6a5bd261eff259df1fbcf198dc4268ed268f6b3abede"}
Sep 11 13:53:47.102891 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.102856    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cgrfd"]
Sep 11 13:53:47.105718 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.105700    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:47.108518 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.108486    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 13:53:47.108683 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.108536    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-swtgt\""
Sep 11 13:53:47.123443 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.123410    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cgrfd"]
Sep 11 13:53:47.192057 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.192021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a7c8a77-7e73-464e-b4d4-e82523801d0e-cert\") pod \"odh-model-controller-7998c67dc4-cgrfd\" (UID: \"0a7c8a77-7e73-464e-b4d4-e82523801d0e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:47.192242 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.192090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z56vt\" (UniqueName: \"kubernetes.io/projected/0a7c8a77-7e73-464e-b4d4-e82523801d0e-kube-api-access-z56vt\") pod \"odh-model-controller-7998c67dc4-cgrfd\" (UID: \"0a7c8a77-7e73-464e-b4d4-e82523801d0e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:47.292608 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.292577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z56vt\" (UniqueName: \"kubernetes.io/projected/0a7c8a77-7e73-464e-b4d4-e82523801d0e-kube-api-access-z56vt\") pod \"odh-model-controller-7998c67dc4-cgrfd\" (UID: \"0a7c8a77-7e73-464e-b4d4-e82523801d0e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:47.292821 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.292797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a7c8a77-7e73-464e-b4d4-e82523801d0e-cert\") pod \"odh-model-controller-7998c67dc4-cgrfd\" (UID: \"0a7c8a77-7e73-464e-b4d4-e82523801d0e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:47.295436 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.295412    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a7c8a77-7e73-464e-b4d4-e82523801d0e-cert\") pod \"odh-model-controller-7998c67dc4-cgrfd\" (UID: \"0a7c8a77-7e73-464e-b4d4-e82523801d0e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:47.301328 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.301298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z56vt\" (UniqueName: \"kubernetes.io/projected/0a7c8a77-7e73-464e-b4d4-e82523801d0e-kube-api-access-z56vt\") pod \"odh-model-controller-7998c67dc4-cgrfd\" (UID: \"0a7c8a77-7e73-464e-b4d4-e82523801d0e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:47.419732 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.419646    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:47.721775 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.721667    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cgrfd"]
Sep 11 13:53:47.741056 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.740877    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6l2mf"]
Sep 11 13:53:47.747838 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.747807    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:47.752896 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.750945    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2fkm9\""
Sep 11 13:53:47.752896 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.751244    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 13:53:47.752896 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.752828    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6l2mf"]
Sep 11 13:53:47.754055 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:53:47.754023    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7c8a77_7e73_464e_b4d4_e82523801d0e.slice/crio-845b14e1c55e2673918d3e74e4880658582396ab1a3f1417c67ec51788d6d9b8 WatchSource:0}: Error finding container 845b14e1c55e2673918d3e74e4880658582396ab1a3f1417c67ec51788d6d9b8: Status 404 returned error can't find the container with id 845b14e1c55e2673918d3e74e4880658582396ab1a3f1417c67ec51788d6d9b8
Sep 11 13:53:47.798329 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.798296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fb5273f-bd3e-4614-9789-2d7f2209d5e1-tls-certs\") pod \"model-serving-api-54d89bd5fc-6l2mf\" (UID: \"9fb5273f-bd3e-4614-9789-2d7f2209d5e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:47.798516 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.798409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmcf2\" (UniqueName: \"kubernetes.io/projected/9fb5273f-bd3e-4614-9789-2d7f2209d5e1-kube-api-access-pmcf2\") pod \"model-serving-api-54d89bd5fc-6l2mf\" (UID: \"9fb5273f-bd3e-4614-9789-2d7f2209d5e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:47.899584 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.899537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fb5273f-bd3e-4614-9789-2d7f2209d5e1-tls-certs\") pod \"model-serving-api-54d89bd5fc-6l2mf\" (UID: \"9fb5273f-bd3e-4614-9789-2d7f2209d5e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:47.899804 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.899598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmcf2\" (UniqueName: \"kubernetes.io/projected/9fb5273f-bd3e-4614-9789-2d7f2209d5e1-kube-api-access-pmcf2\") pod \"model-serving-api-54d89bd5fc-6l2mf\" (UID: \"9fb5273f-bd3e-4614-9789-2d7f2209d5e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:47.902590 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.902558    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fb5273f-bd3e-4614-9789-2d7f2209d5e1-tls-certs\") pod \"model-serving-api-54d89bd5fc-6l2mf\" (UID: \"9fb5273f-bd3e-4614-9789-2d7f2209d5e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:47.908432 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:47.908405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmcf2\" (UniqueName: \"kubernetes.io/projected/9fb5273f-bd3e-4614-9789-2d7f2209d5e1-kube-api-access-pmcf2\") pod \"model-serving-api-54d89bd5fc-6l2mf\" (UID: \"9fb5273f-bd3e-4614-9789-2d7f2209d5e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:48.064136 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:48.064037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:48.343969 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:48.343937    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6l2mf"]
Sep 11 13:53:48.756700 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:48.756657    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd" event={"ID":"0a7c8a77-7e73-464e-b4d4-e82523801d0e","Type":"ContainerStarted","Data":"845b14e1c55e2673918d3e74e4880658582396ab1a3f1417c67ec51788d6d9b8"}
Sep 11 13:53:49.319356 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:53:49.319323    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb5273f_bd3e_4614_9789_2d7f2209d5e1.slice/crio-bacdd144e2059b71176907e86702a702a79ba0aab2f51d7e3559f1a6bd35354d WatchSource:0}: Error finding container bacdd144e2059b71176907e86702a702a79ba0aab2f51d7e3559f1a6bd35354d: Status 404 returned error can't find the container with id bacdd144e2059b71176907e86702a702a79ba0aab2f51d7e3559f1a6bd35354d
Sep 11 13:53:49.356588 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:49.356552    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:49.356673 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:49.356631    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:49.356673 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:49.356663    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:49.766840 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:49.766487    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm" event={"ID":"2b6449a4-e6ad-4bf8-b322-c6b995583ee8","Type":"ContainerStarted","Data":"7627e968be346aafa978151341fc7eb75874e4c580b3bdd68bc564602875cdbf"}
Sep 11 13:53:49.768939 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:49.768901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf" event={"ID":"9fb5273f-bd3e-4614-9789-2d7f2209d5e1","Type":"ContainerStarted","Data":"bacdd144e2059b71176907e86702a702a79ba0aab2f51d7e3559f1a6bd35354d"}
Sep 11 13:53:49.788862 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:49.788373    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm" podStartSLOduration=1.9060892 podStartE2EDuration="4.788350949s" totalImagesPullingTime="2.882261749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:49.786804002 +0000 UTC m=+326.045004011" watchObservedRunningTime="2026-09-11 13:53:49.788350949 +0000 UTC m=+326.046550959"
Sep 11 13:53:50.193430 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:50.193338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:50.199885 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:50.199811    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:50.777343 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:50.777269    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:50.778826 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:50.778740    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm"
Sep 11 13:53:50.828267 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:50.828233    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"]
Sep 11 13:53:53.141567 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.141537    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:53.141955 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.141605    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:53.141955 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.141632    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:53.793502 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.793464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf" event={"ID":"9fb5273f-bd3e-4614-9789-2d7f2209d5e1","Type":"ContainerStarted","Data":"d6bfc35025f1f0dc2f00c5f7c4d3b2950c77dbf8b782b79238ad29c668d64e89"}
Sep 11 13:53:53.793730 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.793569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:53:53.795241 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.795212    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6" event={"ID":"83316f5f-166d-4495-8901-129fdffa0ad9","Type":"ContainerStarted","Data":"d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005"}
Sep 11 13:53:53.795380 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.795307    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6" podUID="83316f5f-166d-4495-8901-129fdffa0ad9" containerName="istio-proxy" containerID="cri-o://d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005" gracePeriod=30
Sep 11 13:53:53.796898 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.796873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd" event={"ID":"0a7c8a77-7e73-464e-b4d4-e82523801d0e","Type":"ContainerStarted","Data":"6bc7c4cac165ecdb0d5df6af86c80d994dc7c767ed02fb5e1e76c815991a212f"}
Sep 11 13:53:53.797021 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.797007    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:53:53.850881 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.850829    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf" podStartSLOduration=3.032262109 podStartE2EDuration="6.850815097s" totalImagesPullingTime="3.818552988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:53.849706224 +0000 UTC m=+330.107906234" watchObservedRunningTime="2026-09-11 13:53:53.850815097 +0000 UTC m=+330.109015106"
Sep 11 13:53:53.900098 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.900043    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd" podStartSLOduration=1.567937974 podStartE2EDuration="6.900024046s" totalImagesPullingTime="5.332086072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:53.898941771 +0000 UTC m=+330.157141780" watchObservedRunningTime="2026-09-11 13:53:53.900024046 +0000 UTC m=+330.158224055"
Sep 11 13:53:53.924257 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:53.924200    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6" podStartSLOduration=1.988744757 podStartE2EDuration="8.924184776s" totalImagesPullingTime="6.935440019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:53.922949736 +0000 UTC m=+330.181149747" watchObservedRunningTime="2026-09-11 13:53:53.924184776 +0000 UTC m=+330.182384785"
Sep 11 13:53:56.827411 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.827369    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"]
Sep 11 13:53:56.831034 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.831010    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:56.834491 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.834464    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 13:53:56.834624 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.834505    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rx5t8\""
Sep 11 13:53:56.846895 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.846870    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"]
Sep 11 13:53:56.887952 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.887912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65cc6a5b-677d-4947-a203-45b4fe333db4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b5cnq\" (UID: \"65cc6a5b-677d-4947-a203-45b4fe333db4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:56.888141 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.888005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhfts\" (UniqueName: \"kubernetes.io/projected/65cc6a5b-677d-4947-a203-45b4fe333db4-kube-api-access-jhfts\") pod \"llmisvc-controller-manager-777c6bd7d5-b5cnq\" (UID: \"65cc6a5b-677d-4947-a203-45b4fe333db4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:56.989316 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.989276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhfts\" (UniqueName: \"kubernetes.io/projected/65cc6a5b-677d-4947-a203-45b4fe333db4-kube-api-access-jhfts\") pod \"llmisvc-controller-manager-777c6bd7d5-b5cnq\" (UID: \"65cc6a5b-677d-4947-a203-45b4fe333db4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:56.989501 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:56.989337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65cc6a5b-677d-4947-a203-45b4fe333db4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b5cnq\" (UID: \"65cc6a5b-677d-4947-a203-45b4fe333db4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:56.989501 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:53:56.989448    2563 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 11 13:53:56.989594 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:53:56.989504    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65cc6a5b-677d-4947-a203-45b4fe333db4-cert podName:65cc6a5b-677d-4947-a203-45b4fe333db4 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:57.489488196 +0000 UTC m=+333.747688183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/65cc6a5b-677d-4947-a203-45b4fe333db4-cert") pod "llmisvc-controller-manager-777c6bd7d5-b5cnq" (UID: "65cc6a5b-677d-4947-a203-45b4fe333db4") : secret "llmisvc-webhook-server-cert" not found
Sep 11 13:53:57.012647 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:57.012608    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhfts\" (UniqueName: \"kubernetes.io/projected/65cc6a5b-677d-4947-a203-45b4fe333db4-kube-api-access-jhfts\") pod \"llmisvc-controller-manager-777c6bd7d5-b5cnq\" (UID: \"65cc6a5b-677d-4947-a203-45b4fe333db4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:57.494014 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:57.493970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65cc6a5b-677d-4947-a203-45b4fe333db4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b5cnq\" (UID: \"65cc6a5b-677d-4947-a203-45b4fe333db4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:57.497496 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:57.497458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65cc6a5b-677d-4947-a203-45b4fe333db4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b5cnq\" (UID: \"65cc6a5b-677d-4947-a203-45b4fe333db4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:57.741649 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:57.741612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:53:58.036597 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:58.036490    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"]
Sep 11 13:53:58.040006 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:53:58.039964    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod65cc6a5b_677d_4947_a203_45b4fe333db4.slice/crio-5990b7a3560301fe246672ddc6f6fd594df52003a2bf4cd2fb1a82c45ae02b84 WatchSource:0}: Error finding container 5990b7a3560301fe246672ddc6f6fd594df52003a2bf4cd2fb1a82c45ae02b84: Status 404 returned error can't find the container with id 5990b7a3560301fe246672ddc6f6fd594df52003a2bf4cd2fb1a82c45ae02b84
Sep 11 13:53:58.824925 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:58.824894    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq" event={"ID":"65cc6a5b-677d-4947-a203-45b4fe333db4","Type":"ContainerStarted","Data":"5990b7a3560301fe246672ddc6f6fd594df52003a2bf4cd2fb1a82c45ae02b84"}
Sep 11 13:53:59.145657 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.145613    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:59.207774 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.207722    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-certs\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.207957 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.207815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-istio-token\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.207957 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.207853    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83316f5f-166d-4495-8901-129fdffa0ad9-istiod-ca-cert\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.207957 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.207882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgzcf\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-kube-api-access-dgzcf\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.208074 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208044    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:59.208166 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208107    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-data\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.208255 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208183    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-socket\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.208255 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-credential-socket\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.208375 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208279    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-envoy\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.208375 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83316f5f-166d-4495-8901-129fdffa0ad9-istio-podinfo\") pod \"83316f5f-166d-4495-8901-129fdffa0ad9\" (UID: \"83316f5f-166d-4495-8901-129fdffa0ad9\") "
Sep 11 13:53:59.208912 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208271    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83316f5f-166d-4495-8901-129fdffa0ad9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:53:59.208912 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208471    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:59.208912 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208596    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.208912 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208616    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.208912 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208630    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83316f5f-166d-4495-8901-129fdffa0ad9-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.208912 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208876    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:59.209356 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.208975    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-data" (OuterVolumeSpecName: "istio-data") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:59.211802 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.211764    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-istio-token" (OuterVolumeSpecName: "istio-token") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:59.211926 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.211838    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-kube-api-access-dgzcf" (OuterVolumeSpecName: "kube-api-access-dgzcf") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "kube-api-access-dgzcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:59.212343 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.212288    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:59.213454 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.213406    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83316f5f-166d-4495-8901-129fdffa0ad9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83316f5f-166d-4495-8901-129fdffa0ad9" (UID: "83316f5f-166d-4495-8901-129fdffa0ad9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:53:59.309678 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.309554    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.309678 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.309581    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgzcf\" (UniqueName: \"kubernetes.io/projected/83316f5f-166d-4495-8901-129fdffa0ad9-kube-api-access-dgzcf\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.309678 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.309649    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.309678 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.309660    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.310111 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.309672    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83316f5f-166d-4495-8901-129fdffa0ad9-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.310111 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.309707    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83316f5f-166d-4495-8901-129fdffa0ad9-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:53:59.831150 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.831113    2563 generic.go:356] "Generic (PLEG): container finished" podID="83316f5f-166d-4495-8901-129fdffa0ad9" containerID="d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005" exitCode=0
Sep 11 13:53:59.831302 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.831165    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"
Sep 11 13:53:59.831302 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.831199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6" event={"ID":"83316f5f-166d-4495-8901-129fdffa0ad9","Type":"ContainerDied","Data":"d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005"}
Sep 11 13:53:59.831302 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.831253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6" event={"ID":"83316f5f-166d-4495-8901-129fdffa0ad9","Type":"ContainerDied","Data":"c91d9ea5491bec4666dca505f94a24ef9e9e6d3dc7768a5807ad409be402cbe1"}
Sep 11 13:53:59.831302 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.831276    2563 scope.go:117] "RemoveContainer" containerID="d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005"
Sep 11 13:53:59.841248 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.841168    2563 scope.go:117] "RemoveContainer" containerID="d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005"
Sep 11 13:53:59.841553 ip-10-0-137-192 kubenswrapper[2563]: E0911 13:53:59.841511    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005\": container with ID starting with d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005 not found: ID does not exist" containerID="d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005"
Sep 11 13:53:59.841737 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.841552    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005"} err="failed to get container status \"d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005\": rpc error: code = NotFound desc = could not find container \"d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005\": container with ID starting with d4159d9dcf5b5e628294f555d432138a2c993098226ca329ee7d064f814b7005 not found: ID does not exist"
Sep 11 13:53:59.854642 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.854613    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"]
Sep 11 13:53:59.860423 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:53:59.860396    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc8m9m6"]
Sep 11 13:54:00.241471 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:00.241435    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83316f5f-166d-4495-8901-129fdffa0ad9" path="/var/lib/kubelet/pods/83316f5f-166d-4495-8901-129fdffa0ad9/volumes"
Sep 11 13:54:00.838582 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:00.838550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq" event={"ID":"65cc6a5b-677d-4947-a203-45b4fe333db4","Type":"ContainerStarted","Data":"dded7a34caad0a80717974e39331fead8fdbb4a6b244f7e4f1027e77ba15ea55"}
Sep 11 13:54:00.838773 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:00.838671    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:54:00.856160 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:00.856106    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq" podStartSLOduration=2.268390563 podStartE2EDuration="4.856089104s" totalImagesPullingTime="2.587698541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:00.85443908 +0000 UTC m=+337.112639088" watchObservedRunningTime="2026-09-11 13:54:00.856089104 +0000 UTC m=+337.114289119"
Sep 11 13:54:04.804607 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:04.804578    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cgrfd"
Sep 11 13:54:04.806737 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:04.806713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6l2mf"
Sep 11 13:54:31.846277 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:31.846244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5cnq"
Sep 11 13:54:52.038807 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.038767    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"]
Sep 11 13:54:52.039305 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.039110    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83316f5f-166d-4495-8901-129fdffa0ad9" containerName="istio-proxy"
Sep 11 13:54:52.039305 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.039124    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="83316f5f-166d-4495-8901-129fdffa0ad9" containerName="istio-proxy"
Sep 11 13:54:52.039305 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.039195    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="83316f5f-166d-4495-8901-129fdffa0ad9" containerName="istio-proxy"
Sep 11 13:54:52.042367 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.042350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.046780 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.045004    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:54:52.046780 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.045967    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:54:52.046780 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.046241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wcl5z\""
Sep 11 13:54:52.051695 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.051657    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"]
Sep 11 13:54:52.137249 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.137216    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"]
Sep 11 13:54:52.140747 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.140730    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.147563 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.147530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjzh7\" (UniqueName: \"kubernetes.io/projected/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-kube-api-access-rjzh7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.147720 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.147592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.147720 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.147654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.149133 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.149111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"]
Sep 11 13:54:52.242274 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.242240    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"]
Sep 11 13:54:52.245576 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.245557    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.248502 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.248478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjzh7\" (UniqueName: \"kubernetes.io/projected/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-kube-api-access-rjzh7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.248687 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.248529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.248687 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.248567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.248687 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.248602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.248687 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.248647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xxmk\" (UniqueName: \"kubernetes.io/projected/ba9fb54c-01ec-4d20-916f-7300e8ad2622-kube-api-access-6xxmk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.248687 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.248681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.248947 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.248928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.249007 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.248990    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.253657 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.253623    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"]
Sep 11 13:54:52.257558 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.257528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjzh7\" (UniqueName: \"kubernetes.io/projected/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-kube-api-access-rjzh7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.339343 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.339260    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"]
Sep 11 13:54:52.342895 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.342877    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.349211 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.349174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xxmk\" (UniqueName: \"kubernetes.io/projected/ba9fb54c-01ec-4d20-916f-7300e8ad2622-kube-api-access-6xxmk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.349369 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.349292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flrbn\" (UniqueName: \"kubernetes.io/projected/81c9246c-e983-4d67-9731-73742e4d79b9-kube-api-access-flrbn\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.349369 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.349335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.349490 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.349376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.349490 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.349401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.349662 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.349556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.349872 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.349846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.350133 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.350109    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.351331 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.351309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"]
Sep 11 13:54:52.357921 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.357898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xxmk\" (UniqueName: \"kubernetes.io/projected/ba9fb54c-01ec-4d20-916f-7300e8ad2622-kube-api-access-6xxmk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.358569 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.358553    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:52.450721 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.450688    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:52.450952 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.450921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.451023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.450962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.451023 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.450998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.451118 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.451034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.451118 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.451110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52stw\" (UniqueName: \"kubernetes.io/projected/4914ac76-0eee-4d30-b81e-af4526af1010-kube-api-access-52stw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.451201 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.451172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flrbn\" (UniqueName: \"kubernetes.io/projected/81c9246c-e983-4d67-9731-73742e4d79b9-kube-api-access-flrbn\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.451324 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.451306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.451382 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.451338    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.460132 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.460107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flrbn\" (UniqueName: \"kubernetes.io/projected/81c9246c-e983-4d67-9731-73742e4d79b9-kube-api-access-flrbn\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.552393 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.552357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52stw\" (UniqueName: \"kubernetes.io/projected/4914ac76-0eee-4d30-b81e-af4526af1010-kube-api-access-52stw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.552614 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.552426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.552614 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.552454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.552929 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.552907    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.552972 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.552930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.556249 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.556235    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:52.564479 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.564451    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52stw\" (UniqueName: \"kubernetes.io/projected/4914ac76-0eee-4d30-b81e-af4526af1010-kube-api-access-52stw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.639832 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.639800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"]
Sep 11 13:54:52.640810 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:54:52.640744    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d7d1e7_2dad_4f69_b3b4_0f4dd7ef2975.slice/crio-2f7550582595cd48ca62648b688f3270b7645d30d76edc4c48e49ddcc09b876b WatchSource:0}: Error finding container 2f7550582595cd48ca62648b688f3270b7645d30d76edc4c48e49ddcc09b876b: Status 404 returned error can't find the container with id 2f7550582595cd48ca62648b688f3270b7645d30d76edc4c48e49ddcc09b876b
Sep 11 13:54:52.653298 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.653274    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:52.729556 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.729524    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"]
Sep 11 13:54:52.730699 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:54:52.730662    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba9fb54c_01ec_4d20_916f_7300e8ad2622.slice/crio-ba9439f1041f9246b6d99d1723f56b4d88340e1ed4fc28b2cf6644868f5bc89f WatchSource:0}: Error finding container ba9439f1041f9246b6d99d1723f56b4d88340e1ed4fc28b2cf6644868f5bc89f: Status 404 returned error can't find the container with id ba9439f1041f9246b6d99d1723f56b4d88340e1ed4fc28b2cf6644868f5bc89f
Sep 11 13:54:52.830498 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.830472    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"]
Sep 11 13:54:52.832494 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:54:52.832466    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c9246c_e983_4d67_9731_73742e4d79b9.slice/crio-1b2b4a7a59f9f2e704af6615e409f948bca44767fe11395577412936f33bc2e3 WatchSource:0}: Error finding container 1b2b4a7a59f9f2e704af6615e409f948bca44767fe11395577412936f33bc2e3: Status 404 returned error can't find the container with id 1b2b4a7a59f9f2e704af6615e409f948bca44767fe11395577412936f33bc2e3
Sep 11 13:54:52.932381 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:52.932348    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"]
Sep 11 13:54:52.935398 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:54:52.935371    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4914ac76_0eee_4d30_b81e_af4526af1010.slice/crio-b0fdd0c30972fc603c16267ab6f06de904d51ab58aad1bfe684d99d0535c0f17 WatchSource:0}: Error finding container b0fdd0c30972fc603c16267ab6f06de904d51ab58aad1bfe684d99d0535c0f17: Status 404 returned error can't find the container with id b0fdd0c30972fc603c16267ab6f06de904d51ab58aad1bfe684d99d0535c0f17
Sep 11 13:54:53.086777 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.086718    2563 generic.go:356] "Generic (PLEG): container finished" podID="4914ac76-0eee-4d30-b81e-af4526af1010" containerID="9c6dc732e390a576ea83279b7762023ae7834eeb0f37589954ba901884e4d7a3" exitCode=0
Sep 11 13:54:53.087252 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.086807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk" event={"ID":"4914ac76-0eee-4d30-b81e-af4526af1010","Type":"ContainerDied","Data":"9c6dc732e390a576ea83279b7762023ae7834eeb0f37589954ba901884e4d7a3"}
Sep 11 13:54:53.087252 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.086862    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk" event={"ID":"4914ac76-0eee-4d30-b81e-af4526af1010","Type":"ContainerStarted","Data":"b0fdd0c30972fc603c16267ab6f06de904d51ab58aad1bfe684d99d0535c0f17"}
Sep 11 13:54:53.088577 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.088555    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerID="d0477935b0729ec12d71ef8d6d0d0e91d0cf551fdc44a00cff8e3cd8da52b8e7" exitCode=0
Sep 11 13:54:53.088647 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.088631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc" event={"ID":"ba9fb54c-01ec-4d20-916f-7300e8ad2622","Type":"ContainerDied","Data":"d0477935b0729ec12d71ef8d6d0d0e91d0cf551fdc44a00cff8e3cd8da52b8e7"}
Sep 11 13:54:53.088698 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.088681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc" event={"ID":"ba9fb54c-01ec-4d20-916f-7300e8ad2622","Type":"ContainerStarted","Data":"ba9439f1041f9246b6d99d1723f56b4d88340e1ed4fc28b2cf6644868f5bc89f"}
Sep 11 13:54:53.090479 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.090449    2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerID="74b604b0864647ec40612105b9975912627222e02f40ac81ca7666cd13471744" exitCode=0
Sep 11 13:54:53.090564 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.090540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4" event={"ID":"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975","Type":"ContainerDied","Data":"74b604b0864647ec40612105b9975912627222e02f40ac81ca7666cd13471744"}
Sep 11 13:54:53.090627 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.090607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4" event={"ID":"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975","Type":"ContainerStarted","Data":"2f7550582595cd48ca62648b688f3270b7645d30d76edc4c48e49ddcc09b876b"}
Sep 11 13:54:53.092378 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.092362    2563 generic.go:356] "Generic (PLEG): container finished" podID="81c9246c-e983-4d67-9731-73742e4d79b9" containerID="0b9343bf1aed440194b1271f130b9ae17b4846810b89308ea6e187daf78e5113" exitCode=0
Sep 11 13:54:53.092445 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.092401    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp" event={"ID":"81c9246c-e983-4d67-9731-73742e4d79b9","Type":"ContainerDied","Data":"0b9343bf1aed440194b1271f130b9ae17b4846810b89308ea6e187daf78e5113"}
Sep 11 13:54:53.092445 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:53.092428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp" event={"ID":"81c9246c-e983-4d67-9731-73742e4d79b9","Type":"ContainerStarted","Data":"1b2b4a7a59f9f2e704af6615e409f948bca44767fe11395577412936f33bc2e3"}
Sep 11 13:54:54.108873 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:54.108838    2563 generic.go:356] "Generic (PLEG): container finished" podID="81c9246c-e983-4d67-9731-73742e4d79b9" containerID="428867d863c44795db72f51b25f47d9beb8458d4f35239c8b76d39be034d8eec" exitCode=0
Sep 11 13:54:54.109308 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:54.108921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp" event={"ID":"81c9246c-e983-4d67-9731-73742e4d79b9","Type":"ContainerDied","Data":"428867d863c44795db72f51b25f47d9beb8458d4f35239c8b76d39be034d8eec"}
Sep 11 13:54:54.111252 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:54.111226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc" event={"ID":"ba9fb54c-01ec-4d20-916f-7300e8ad2622","Type":"ContainerStarted","Data":"ac3447696162d12badf6171195612a1d9f0274be33c7528dc1eba5ba41f05977"}
Sep 11 13:54:55.121207 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:55.121174    2563 generic.go:356] "Generic (PLEG): container finished" podID="4914ac76-0eee-4d30-b81e-af4526af1010" containerID="4437fb31a436fde467ae5683117ab7eb00168db6389581bb26dbeb52fd34480e" exitCode=0
Sep 11 13:54:55.121671 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:55.121251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk" event={"ID":"4914ac76-0eee-4d30-b81e-af4526af1010","Type":"ContainerDied","Data":"4437fb31a436fde467ae5683117ab7eb00168db6389581bb26dbeb52fd34480e"}
Sep 11 13:54:55.123338 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:55.123303    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerID="ac3447696162d12badf6171195612a1d9f0274be33c7528dc1eba5ba41f05977" exitCode=0
Sep 11 13:54:55.123420 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:55.123359    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc" event={"ID":"ba9fb54c-01ec-4d20-916f-7300e8ad2622","Type":"ContainerDied","Data":"ac3447696162d12badf6171195612a1d9f0274be33c7528dc1eba5ba41f05977"}
Sep 11 13:54:55.125733 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:55.125710    2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerID="d6e9ee5917103b990666370a236459e72e0bb70ff9efb9781331518c17810e76" exitCode=0
Sep 11 13:54:55.125856 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:55.125787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4" event={"ID":"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975","Type":"ContainerDied","Data":"d6e9ee5917103b990666370a236459e72e0bb70ff9efb9781331518c17810e76"}
Sep 11 13:54:55.128495 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:55.128476    2563 generic.go:356] "Generic (PLEG): container finished" podID="81c9246c-e983-4d67-9731-73742e4d79b9" containerID="e81ce730b1d0bfc253068ec4f36282ff2a1d0127adc6de12330e4fc39a8a551c" exitCode=0
Sep 11 13:54:55.128571 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:55.128541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp" event={"ID":"81c9246c-e983-4d67-9731-73742e4d79b9","Type":"ContainerDied","Data":"e81ce730b1d0bfc253068ec4f36282ff2a1d0127adc6de12330e4fc39a8a551c"}
Sep 11 13:54:56.135738 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.135702    2563 generic.go:356] "Generic (PLEG): container finished" podID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerID="85e0041b969e1872ab768bbd21358a40fab582b66557a58f988dd82cc31addb3" exitCode=0
Sep 11 13:54:56.136212 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.135851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4" event={"ID":"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975","Type":"ContainerDied","Data":"85e0041b969e1872ab768bbd21358a40fab582b66557a58f988dd82cc31addb3"}
Sep 11 13:54:56.138161 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.138134    2563 generic.go:356] "Generic (PLEG): container finished" podID="4914ac76-0eee-4d30-b81e-af4526af1010" containerID="d94dc264a3897f6398b7701af20d9e486aef33735466d58bd78d0f44c1ee705b" exitCode=0
Sep 11 13:54:56.138282 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.138213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk" event={"ID":"4914ac76-0eee-4d30-b81e-af4526af1010","Type":"ContainerDied","Data":"d94dc264a3897f6398b7701af20d9e486aef33735466d58bd78d0f44c1ee705b"}
Sep 11 13:54:56.140420 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.140397    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerID="15f3c68cd0066fea3ce02d1f0fbd2fc81d23cd8f335a3878bb7575fabd96e24b" exitCode=0
Sep 11 13:54:56.140531 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.140460    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc" event={"ID":"ba9fb54c-01ec-4d20-916f-7300e8ad2622","Type":"ContainerDied","Data":"15f3c68cd0066fea3ce02d1f0fbd2fc81d23cd8f335a3878bb7575fabd96e24b"}
Sep 11 13:54:56.282351 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.282326    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:56.387501 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.387412    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-util\") pod \"81c9246c-e983-4d67-9731-73742e4d79b9\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") "
Sep 11 13:54:56.387501 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.387486    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flrbn\" (UniqueName: \"kubernetes.io/projected/81c9246c-e983-4d67-9731-73742e4d79b9-kube-api-access-flrbn\") pod \"81c9246c-e983-4d67-9731-73742e4d79b9\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") "
Sep 11 13:54:56.387739 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.387509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-bundle\") pod \"81c9246c-e983-4d67-9731-73742e4d79b9\" (UID: \"81c9246c-e983-4d67-9731-73742e4d79b9\") "
Sep 11 13:54:56.388197 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.388169    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-bundle" (OuterVolumeSpecName: "bundle") pod "81c9246c-e983-4d67-9731-73742e4d79b9" (UID: "81c9246c-e983-4d67-9731-73742e4d79b9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:54:56.389691 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.389667    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c9246c-e983-4d67-9731-73742e4d79b9-kube-api-access-flrbn" (OuterVolumeSpecName: "kube-api-access-flrbn") pod "81c9246c-e983-4d67-9731-73742e4d79b9" (UID: "81c9246c-e983-4d67-9731-73742e4d79b9"). InnerVolumeSpecName "kube-api-access-flrbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:54:56.392786 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.392709    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-util" (OuterVolumeSpecName: "util") pod "81c9246c-e983-4d67-9731-73742e4d79b9" (UID: "81c9246c-e983-4d67-9731-73742e4d79b9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:54:56.488576 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.488534    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-util\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:56.488576 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.488568    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flrbn\" (UniqueName: \"kubernetes.io/projected/81c9246c-e983-4d67-9731-73742e4d79b9-kube-api-access-flrbn\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:56.488576 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:56.488580    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81c9246c-e983-4d67-9731-73742e4d79b9-bundle\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.147517 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.147476    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp"
Sep 11 13:54:57.147517 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.147472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atljbp" event={"ID":"81c9246c-e983-4d67-9731-73742e4d79b9","Type":"ContainerDied","Data":"1b2b4a7a59f9f2e704af6615e409f948bca44767fe11395577412936f33bc2e3"}
Sep 11 13:54:57.148003 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.147531    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b2b4a7a59f9f2e704af6615e409f948bca44767fe11395577412936f33bc2e3"
Sep 11 13:54:57.333718 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.333692    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:57.335771 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.335736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:57.338041 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.338023    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:54:57.396161 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-util\") pod \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") "
Sep 11 13:54:57.396359 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396189    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xxmk\" (UniqueName: \"kubernetes.io/projected/ba9fb54c-01ec-4d20-916f-7300e8ad2622-kube-api-access-6xxmk\") pod \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") "
Sep 11 13:54:57.396359 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-bundle\") pod \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") "
Sep 11 13:54:57.396359 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396253    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-util\") pod \"4914ac76-0eee-4d30-b81e-af4526af1010\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") "
Sep 11 13:54:57.396359 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396275    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjzh7\" (UniqueName: \"kubernetes.io/projected/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-kube-api-access-rjzh7\") pod \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") "
Sep 11 13:54:57.396359 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396317    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-util\") pod \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\" (UID: \"ba9fb54c-01ec-4d20-916f-7300e8ad2622\") "
Sep 11 13:54:57.396359 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396345    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-bundle\") pod \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\" (UID: \"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975\") "
Sep 11 13:54:57.396676 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396372    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52stw\" (UniqueName: \"kubernetes.io/projected/4914ac76-0eee-4d30-b81e-af4526af1010-kube-api-access-52stw\") pod \"4914ac76-0eee-4d30-b81e-af4526af1010\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") "
Sep 11 13:54:57.396676 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.396410    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-bundle\") pod \"4914ac76-0eee-4d30-b81e-af4526af1010\" (UID: \"4914ac76-0eee-4d30-b81e-af4526af1010\") "
Sep 11 13:54:57.397464 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.397206    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-bundle" (OuterVolumeSpecName: "bundle") pod "ba9fb54c-01ec-4d20-916f-7300e8ad2622" (UID: "ba9fb54c-01ec-4d20-916f-7300e8ad2622"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:54:57.397633 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.397559    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-bundle" (OuterVolumeSpecName: "bundle") pod "4914ac76-0eee-4d30-b81e-af4526af1010" (UID: "4914ac76-0eee-4d30-b81e-af4526af1010"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:54:57.398032 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.397981    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-bundle" (OuterVolumeSpecName: "bundle") pod "c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" (UID: "c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:54:57.398978 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.398938    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba9fb54c-01ec-4d20-916f-7300e8ad2622-kube-api-access-6xxmk" (OuterVolumeSpecName: "kube-api-access-6xxmk") pod "ba9fb54c-01ec-4d20-916f-7300e8ad2622" (UID: "ba9fb54c-01ec-4d20-916f-7300e8ad2622"). InnerVolumeSpecName "kube-api-access-6xxmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:54:57.400438 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.400404    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-kube-api-access-rjzh7" (OuterVolumeSpecName: "kube-api-access-rjzh7") pod "c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" (UID: "c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975"). InnerVolumeSpecName "kube-api-access-rjzh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:54:57.401820 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.401789    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4914ac76-0eee-4d30-b81e-af4526af1010-kube-api-access-52stw" (OuterVolumeSpecName: "kube-api-access-52stw") pod "4914ac76-0eee-4d30-b81e-af4526af1010" (UID: "4914ac76-0eee-4d30-b81e-af4526af1010"). InnerVolumeSpecName "kube-api-access-52stw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:54:57.404221 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.404193    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-util" (OuterVolumeSpecName: "util") pod "c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" (UID: "c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:54:57.404741 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.404714    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-util" (OuterVolumeSpecName: "util") pod "ba9fb54c-01ec-4d20-916f-7300e8ad2622" (UID: "ba9fb54c-01ec-4d20-916f-7300e8ad2622"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:54:57.405159 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.405135    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-util" (OuterVolumeSpecName: "util") pod "4914ac76-0eee-4d30-b81e-af4526af1010" (UID: "4914ac76-0eee-4d30-b81e-af4526af1010"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:54:57.497498 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497454    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xxmk\" (UniqueName: \"kubernetes.io/projected/ba9fb54c-01ec-4d20-916f-7300e8ad2622-kube-api-access-6xxmk\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.497498 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497491    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-bundle\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.497498 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497501    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-util\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.497498 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497509    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjzh7\" (UniqueName: \"kubernetes.io/projected/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-kube-api-access-rjzh7\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.497823 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497518    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba9fb54c-01ec-4d20-916f-7300e8ad2622-util\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.497823 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497526    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-bundle\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.497823 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497533    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52stw\" (UniqueName: \"kubernetes.io/projected/4914ac76-0eee-4d30-b81e-af4526af1010-kube-api-access-52stw\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.497823 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497541    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4914ac76-0eee-4d30-b81e-af4526af1010-bundle\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:57.497823 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:57.497549    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975-util\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 13:54:58.155625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.155586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk" event={"ID":"4914ac76-0eee-4d30-b81e-af4526af1010","Type":"ContainerDied","Data":"b0fdd0c30972fc603c16267ab6f06de904d51ab58aad1bfe684d99d0535c0f17"}
Sep 11 13:54:58.156344 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.155642    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0fdd0c30972fc603c16267ab6f06de904d51ab58aad1bfe684d99d0535c0f17"
Sep 11 13:54:58.156344 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.155598    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czjslk"
Sep 11 13:54:58.157723 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.157695    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc"
Sep 11 13:54:58.157723 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.157710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ht7sc" event={"ID":"ba9fb54c-01ec-4d20-916f-7300e8ad2622","Type":"ContainerDied","Data":"ba9439f1041f9246b6d99d1723f56b4d88340e1ed4fc28b2cf6644868f5bc89f"}
Sep 11 13:54:58.157933 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.157780    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba9439f1041f9246b6d99d1723f56b4d88340e1ed4fc28b2cf6644868f5bc89f"
Sep 11 13:54:58.159802 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.159778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4" event={"ID":"c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975","Type":"ContainerDied","Data":"2f7550582595cd48ca62648b688f3270b7645d30d76edc4c48e49ddcc09b876b"}
Sep 11 13:54:58.159910 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.159824    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f7550582595cd48ca62648b688f3270b7645d30d76edc4c48e49ddcc09b876b"
Sep 11 13:54:58.159910 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:54:58.159829    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wctr4"
Sep 11 13:55:09.476232 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476194    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-df96r"]
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476543    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4914ac76-0eee-4d30-b81e-af4526af1010" containerName="util"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476555    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4914ac76-0eee-4d30-b81e-af4526af1010" containerName="util"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476565    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerName="util"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476570    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerName="util"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476579    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerName="extract"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476585    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerName="extract"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476595    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81c9246c-e983-4d67-9731-73742e4d79b9" containerName="util"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476600    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81c9246c-e983-4d67-9731-73742e4d79b9" containerName="util"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476607    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4914ac76-0eee-4d30-b81e-af4526af1010" containerName="pull"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476612    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4914ac76-0eee-4d30-b81e-af4526af1010" containerName="pull"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476623    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81c9246c-e983-4d67-9731-73742e4d79b9" containerName="extract"
Sep 11 13:55:09.476625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476628    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81c9246c-e983-4d67-9731-73742e4d79b9" containerName="extract"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476633    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerName="pull"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476639    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerName="pull"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476646    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerName="util"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476651    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerName="util"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476656    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81c9246c-e983-4d67-9731-73742e4d79b9" containerName="pull"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476661    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81c9246c-e983-4d67-9731-73742e4d79b9" containerName="pull"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476665    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerName="pull"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476670    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerName="pull"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476676    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4914ac76-0eee-4d30-b81e-af4526af1010" containerName="extract"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476680    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4914ac76-0eee-4d30-b81e-af4526af1010" containerName="extract"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476685    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerName="extract"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476689    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerName="extract"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476740    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba9fb54c-01ec-4d20-916f-7300e8ad2622" containerName="extract"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476765    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4914ac76-0eee-4d30-b81e-af4526af1010" containerName="extract"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476774    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d7d1e7-2dad-4f69-b3b4-0f4dd7ef2975" containerName="extract"
Sep 11 13:55:09.477027 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.476783    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="81c9246c-e983-4d67-9731-73742e4d79b9" containerName="extract"
Sep 11 13:55:09.481254 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.481234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-df96r"
Sep 11 13:55:09.486680 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.486651    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-dpj8n\""
Sep 11 13:55:09.486867 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.486704    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 13:55:09.486966 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.486947    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 13:55:09.496040 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.496010    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-df96r"]
Sep 11 13:55:09.612932 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.612878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pz2s\" (UniqueName: \"kubernetes.io/projected/bf708184-4e4f-426c-9115-4e230f40b1fe-kube-api-access-7pz2s\") pod \"authorino-operator-76b478bd78-df96r\" (UID: \"bf708184-4e4f-426c-9115-4e230f40b1fe\") " pod="kuadrant-system/authorino-operator-76b478bd78-df96r"
Sep 11 13:55:09.714380 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.714334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pz2s\" (UniqueName: \"kubernetes.io/projected/bf708184-4e4f-426c-9115-4e230f40b1fe-kube-api-access-7pz2s\") pod \"authorino-operator-76b478bd78-df96r\" (UID: \"bf708184-4e4f-426c-9115-4e230f40b1fe\") " pod="kuadrant-system/authorino-operator-76b478bd78-df96r"
Sep 11 13:55:09.727817 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.727735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pz2s\" (UniqueName: \"kubernetes.io/projected/bf708184-4e4f-426c-9115-4e230f40b1fe-kube-api-access-7pz2s\") pod \"authorino-operator-76b478bd78-df96r\" (UID: \"bf708184-4e4f-426c-9115-4e230f40b1fe\") " pod="kuadrant-system/authorino-operator-76b478bd78-df96r"
Sep 11 13:55:09.791615 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:09.791580    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-df96r"
Sep 11 13:55:10.078613 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:10.078589    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-df96r"]
Sep 11 13:55:10.083921 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:55:10.083882    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf708184_4e4f_426c_9115_4e230f40b1fe.slice/crio-d84537c31e42c365f3de8f5a827bdb68dc57a3208edd2e5363a1a969da39c4cf WatchSource:0}: Error finding container d84537c31e42c365f3de8f5a827bdb68dc57a3208edd2e5363a1a969da39c4cf: Status 404 returned error can't find the container with id d84537c31e42c365f3de8f5a827bdb68dc57a3208edd2e5363a1a969da39c4cf
Sep 11 13:55:10.222650 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:10.222617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-df96r" event={"ID":"bf708184-4e4f-426c-9115-4e230f40b1fe","Type":"ContainerStarted","Data":"d84537c31e42c365f3de8f5a827bdb68dc57a3208edd2e5363a1a969da39c4cf"}
Sep 11 13:55:13.241364 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:13.241330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-df96r" event={"ID":"bf708184-4e4f-426c-9115-4e230f40b1fe","Type":"ContainerStarted","Data":"b3d0dc7636fb3b8a127c4bc2b57c7feb8edbd5e7d91efdf78fcec6b58c1ff612"}
Sep 11 13:55:13.241833 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:13.241452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-df96r"
Sep 11 13:55:13.259151 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:13.259094    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-df96r" podStartSLOduration=1.812854537 podStartE2EDuration="4.259077297s" totalImagesPullingTime="2.44622276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:13.258246896 +0000 UTC m=+409.516446907" watchObservedRunningTime="2026-09-11 13:55:13.259077297 +0000 UTC m=+409.517277305"
Sep 11 13:55:14.715888 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.715845    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"]
Sep 11 13:55:14.719364 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.719346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.722362 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.722336    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 13:55:14.722362 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.722358    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xzr8s\""
Sep 11 13:55:14.722650 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.722394    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 13:55:14.726610 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.726589    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"]
Sep 11 13:55:14.867818 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.867738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6cc99932-beba-4190-a0fd-8748cb766402-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.868010 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.867857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cc99932-beba-4190-a0fd-8748cb766402-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.868010 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.867922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pl6s\" (UniqueName: \"kubernetes.io/projected/6cc99932-beba-4190-a0fd-8748cb766402-kube-api-access-5pl6s\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.968535 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.968440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cc99932-beba-4190-a0fd-8748cb766402-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.968535 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.968500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pl6s\" (UniqueName: \"kubernetes.io/projected/6cc99932-beba-4190-a0fd-8748cb766402-kube-api-access-5pl6s\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.968736 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.968571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6cc99932-beba-4190-a0fd-8748cb766402-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.969279 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.969252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6cc99932-beba-4190-a0fd-8748cb766402-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.971240 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.971219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cc99932-beba-4190-a0fd-8748cb766402-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:14.979745 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:14.979719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pl6s\" (UniqueName: \"kubernetes.io/projected/6cc99932-beba-4190-a0fd-8748cb766402-kube-api-access-5pl6s\") pod \"kuadrant-console-plugin-fbdd4f445-vrxch\" (UID: \"6cc99932-beba-4190-a0fd-8748cb766402\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:15.030408 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:15.030368    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"
Sep 11 13:55:15.304561 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:15.304538    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch"]
Sep 11 13:55:15.306814 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:55:15.306790    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cc99932_beba_4190_a0fd_8748cb766402.slice/crio-f587261a2a1657fedc3723c55f3a87c69ca32e74f5862293dafdb14e12f135fd WatchSource:0}: Error finding container f587261a2a1657fedc3723c55f3a87c69ca32e74f5862293dafdb14e12f135fd: Status 404 returned error can't find the container with id f587261a2a1657fedc3723c55f3a87c69ca32e74f5862293dafdb14e12f135fd
Sep 11 13:55:16.258113 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:16.258068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch" event={"ID":"6cc99932-beba-4190-a0fd-8748cb766402","Type":"ContainerStarted","Data":"f587261a2a1657fedc3723c55f3a87c69ca32e74f5862293dafdb14e12f135fd"}
Sep 11 13:55:24.250242 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:24.250202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-df96r"
Sep 11 13:55:27.320543 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:27.320498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch" event={"ID":"6cc99932-beba-4190-a0fd-8748cb766402","Type":"ContainerStarted","Data":"2a9f649c28ff5f5aa6e4afb9fb1f81c37817802162ed0dd3078a26cf9241a82f"}
Sep 11 13:55:27.337663 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:27.337605    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vrxch" podStartSLOduration=2.289787983 podStartE2EDuration="13.337591125s" totalImagesPullingTime="11.047803142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:27.337323236 +0000 UTC m=+423.595523247" watchObservedRunningTime="2026-09-11 13:55:27.337591125 +0000 UTC m=+423.595791135"
Sep 11 13:55:54.318365 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.318320    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"]
Sep 11 13:55:54.322599 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.322570    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.325444 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.325416    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zbkkq\""
Sep 11 13:55:54.337505 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.337472    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"]
Sep 11 13:55:54.374130 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.374303 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.374303 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.374303 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.374303 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22dwb\" (UniqueName: \"kubernetes.io/projected/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-kube-api-access-22dwb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.374439 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.374439 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.374439 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.374537 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.374446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.475561 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.475526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.475561 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.475562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.475858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.475580    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22dwb\" (UniqueName: \"kubernetes.io/projected/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-kube-api-access-22dwb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.475858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.475801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.475858 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.475843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476024 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.475941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476024 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.475985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476131 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.476039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476131 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.476093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476259 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.476233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476439 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.476412    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476516 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.476423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476572 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.476539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.476654 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.476635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.478269 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.478249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.478528 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.478510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.484363 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.484333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22dwb\" (UniqueName: \"kubernetes.io/projected/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-kube-api-access-22dwb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.484484 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.484463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8253d448-24ad-4ad7-9cf6-3ec341ef5bdf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-c2426\" (UID: \"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.634022 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.633993    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:54.924777 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.924731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"]
Sep 11 13:55:54.926414 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:55:54.926383    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8253d448_24ad_4ad7_9cf6_3ec341ef5bdf.slice/crio-6b1720a5057ea2cfc7b045a89bd98135fd661f1b1b9137952fb3227469a0bc73 WatchSource:0}: Error finding container 6b1720a5057ea2cfc7b045a89bd98135fd661f1b1b9137952fb3227469a0bc73: Status 404 returned error can't find the container with id 6b1720a5057ea2cfc7b045a89bd98135fd661f1b1b9137952fb3227469a0bc73
Sep 11 13:55:54.928489 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.928458    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:55:54.928550 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.928539    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:55:54.928588 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:54.928569    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:55:55.466204 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:55.466171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" event={"ID":"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf","Type":"ContainerStarted","Data":"49bd82dee156235d982e7497f7fe93e515c0ae1418b29b18123abc9393c6d3c0"}
Sep 11 13:55:55.466626 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:55.466219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" event={"ID":"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf","Type":"ContainerStarted","Data":"6b1720a5057ea2cfc7b045a89bd98135fd661f1b1b9137952fb3227469a0bc73"}
Sep 11 13:55:55.488093 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:55.488048    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" podStartSLOduration=1.488031987 podStartE2EDuration="1.488031987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:55.486945258 +0000 UTC m=+451.745145269" watchObservedRunningTime="2026-09-11 13:55:55.488031987 +0000 UTC m=+451.746231996"
Sep 11 13:55:55.634991 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:55.634949    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:55.640293 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:55.640262    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:56.471747 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.471712    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:56.472896 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.472879    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:55:56.708256 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.708222    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:55:56.711659 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.711640    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:55:56.714250 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.714228    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 13:55:56.720221 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.720193    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:55:56.749516 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.749428    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:55:56.798699 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.798670    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a409f61-fb88-4581-a8d8-937b7a304931-config-file\") pod \"limitador-limitador-84994f7ffc-wgs8h\" (UID: \"6a409f61-fb88-4581-a8d8-937b7a304931\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:55:56.798888 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.798812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxkgh\" (UniqueName: \"kubernetes.io/projected/6a409f61-fb88-4581-a8d8-937b7a304931-kube-api-access-cxkgh\") pod \"limitador-limitador-84994f7ffc-wgs8h\" (UID: \"6a409f61-fb88-4581-a8d8-937b7a304931\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:55:56.900006 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.899971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxkgh\" (UniqueName: \"kubernetes.io/projected/6a409f61-fb88-4581-a8d8-937b7a304931-kube-api-access-cxkgh\") pod \"limitador-limitador-84994f7ffc-wgs8h\" (UID: \"6a409f61-fb88-4581-a8d8-937b7a304931\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:55:56.900006 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.900021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a409f61-fb88-4581-a8d8-937b7a304931-config-file\") pod \"limitador-limitador-84994f7ffc-wgs8h\" (UID: \"6a409f61-fb88-4581-a8d8-937b7a304931\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:55:56.900670 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.900650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a409f61-fb88-4581-a8d8-937b7a304931-config-file\") pod \"limitador-limitador-84994f7ffc-wgs8h\" (UID: \"6a409f61-fb88-4581-a8d8-937b7a304931\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:55:56.908345 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:56.908317    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxkgh\" (UniqueName: \"kubernetes.io/projected/6a409f61-fb88-4581-a8d8-937b7a304931-kube-api-access-cxkgh\") pod \"limitador-limitador-84994f7ffc-wgs8h\" (UID: \"6a409f61-fb88-4581-a8d8-937b7a304931\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:55:57.024575 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:57.024473    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:55:57.325715 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:57.322169    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:55:57.326873 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:55:57.326844    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a409f61_fb88_4581_a8d8_937b7a304931.slice/crio-953414cc7cb501010af89ddb63c77bd1c99f2e19316462b8ca1b9c5a7256b45e WatchSource:0}: Error finding container 953414cc7cb501010af89ddb63c77bd1c99f2e19316462b8ca1b9c5a7256b45e: Status 404 returned error can't find the container with id 953414cc7cb501010af89ddb63c77bd1c99f2e19316462b8ca1b9c5a7256b45e
Sep 11 13:55:57.478185 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:55:57.478147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h" event={"ID":"6a409f61-fb88-4581-a8d8-937b7a304931","Type":"ContainerStarted","Data":"953414cc7cb501010af89ddb63c77bd1c99f2e19316462b8ca1b9c5a7256b45e"}
Sep 11 13:56:16.577634 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:56:16.577525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h" event={"ID":"6a409f61-fb88-4581-a8d8-937b7a304931","Type":"ContainerStarted","Data":"e33d0105eeb82b1bef7110a3c75bfceba8c6d34030a64eb573a66bf1be29df0e"}
Sep 11 13:56:16.577634 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:56:16.577596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:56:16.596522 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:56:16.596468    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h" podStartSLOduration=1.741311125 podStartE2EDuration="20.596452275s" totalImagesPullingTime="18.85514115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:56:16.595160385 +0000 UTC m=+472.853360393" watchObservedRunningTime="2026-09-11 13:56:16.596452275 +0000 UTC m=+472.854652284"
Sep 11 13:56:27.583701 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:56:27.583670    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"
Sep 11 13:57:30.925988 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:30.925898    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:57:41.056188 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:41.056151    2563 generic.go:356] "Generic (PLEG): container finished" podID="8253d448-24ad-4ad7-9cf6-3ec341ef5bdf" containerID="49bd82dee156235d982e7497f7fe93e515c0ae1418b29b18123abc9393c6d3c0" exitCode=0
Sep 11 13:57:41.056615 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:41.056228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" event={"ID":"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf","Type":"ContainerDied","Data":"49bd82dee156235d982e7497f7fe93e515c0ae1418b29b18123abc9393c6d3c0"}
Sep 11 13:57:41.056659 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:41.056620    2563 scope.go:117] "RemoveContainer" containerID="49bd82dee156235d982e7497f7fe93e515c0ae1418b29b18123abc9393c6d3c0"
Sep 11 13:57:41.057671 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:41.057645    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:57:41.057726 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:41.057714    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:57:41.057797 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:41.057785    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:57:41.471887 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:41.471840    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:57:42.064101 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:42.064067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" event={"ID":"8253d448-24ad-4ad7-9cf6-3ec341ef5bdf","Type":"ContainerStarted","Data":"a6d8f4006bc856d646958240ceaa8d311dc225cca89b968923c79551e4751485"}
Sep 11 13:57:42.634197 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:42.634161    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:57:42.635337 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:42.635310    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-c2426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 11 13:57:42.635480 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:42.635362    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" podUID="8253d448-24ad-4ad7-9cf6-3ec341ef5bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 11 13:57:43.635459 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:43.635418    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-c2426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 11 13:57:43.635941 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:43.635478    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" podUID="8253d448-24ad-4ad7-9cf6-3ec341ef5bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 11 13:57:44.635182 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:44.635148    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-c2426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 11 13:57:44.635388 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:44.635304    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" podUID="8253d448-24ad-4ad7-9cf6-3ec341ef5bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 11 13:57:45.634635 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:45.634598    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-c2426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 11 13:57:45.635083 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:45.634654    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" podUID="8253d448-24ad-4ad7-9cf6-3ec341ef5bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 11 13:57:46.635258 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:46.635214    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-c2426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 11 13:57:46.635675 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:46.635278    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426" podUID="8253d448-24ad-4ad7-9cf6-3ec341ef5bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 11 13:57:47.639304 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:47.639273    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:57:47.639777 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:47.639566    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:57:47.640227 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:47.640207    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-c2426"
Sep 11 13:57:53.731336 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:57:53.731301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:58:09.541472 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:58:09.541428    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:58:24.232185 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:58:24.232155    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 13:58:24.232708 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:58:24.232686    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 13:58:50.977373 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:58:50.977283    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:59:07.635744 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:07.635706    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:59:17.040507 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:17.040462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:59:34.672255 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.672217    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"]
Sep 11 13:59:34.676513 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.676489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.681235 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.680969    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:59:34.681235 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.681130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 11 13:59:34.681463 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.681445    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:59:34.681625 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.681595    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p2lbh\""
Sep 11 13:59:34.684936 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.684908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"]
Sep 11 13:59:34.812088 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.812053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.812265 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.812103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07242df5-b0d4-4049-bd30-630fe493b02d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.812265 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.812144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.812265 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.812186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6jxw\" (UniqueName: \"kubernetes.io/projected/07242df5-b0d4-4049-bd30-630fe493b02d-kube-api-access-r6jxw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.812265 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.812238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.812408 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.812276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.913441 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.913441 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913449    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.913714 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07242df5-b0d4-4049-bd30-630fe493b02d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.913714 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.913714 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6jxw\" (UniqueName: \"kubernetes.io/projected/07242df5-b0d4-4049-bd30-630fe493b02d-kube-api-access-r6jxw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.913714 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.913990 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.914053 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.914053 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.913977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.916074 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.916038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07242df5-b0d4-4049-bd30-630fe493b02d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.916316 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.916298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07242df5-b0d4-4049-bd30-630fe493b02d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.922724 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.922666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6jxw\" (UniqueName: \"kubernetes.io/projected/07242df5-b0d4-4049-bd30-630fe493b02d-kube-api-access-r6jxw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-szh5n\" (UID: \"07242df5-b0d4-4049-bd30-630fe493b02d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:34.989660 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:34.989612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:35.269929 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:35.269902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"]
Sep 11 13:59:35.271642 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:59:35.271603    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07242df5_b0d4_4049_bd30_630fe493b02d.slice/crio-c59a6ad7404cfa48b39dcb738540b077b32f13fd8f300d05c87080e668918e16 WatchSource:0}: Error finding container c59a6ad7404cfa48b39dcb738540b077b32f13fd8f300d05c87080e668918e16: Status 404 returned error can't find the container with id c59a6ad7404cfa48b39dcb738540b077b32f13fd8f300d05c87080e668918e16
Sep 11 13:59:35.273094 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:35.273075    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:59:35.697601 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:35.697558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n" event={"ID":"07242df5-b0d4-4049-bd30-630fe493b02d","Type":"ContainerStarted","Data":"c59a6ad7404cfa48b39dcb738540b077b32f13fd8f300d05c87080e668918e16"}
Sep 11 13:59:36.436168 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:36.436133    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:59:36.704714 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:36.704673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n" event={"ID":"07242df5-b0d4-4049-bd30-630fe493b02d","Type":"ContainerStarted","Data":"70abfa30c8eb628ef72f95edbd0f5aecfe8cc08a131169e4a138b8ce08ce927b"}
Sep 11 13:59:36.723805 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:36.723744    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n" podStartSLOduration=1.73555693 podStartE2EDuration="2.723725654s" totalImagesPullingTime="988.168724ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:36.722449007 +0000 UTC m=+672.980649017" watchObservedRunningTime="2026-09-11 13:59:36.723725654 +0000 UTC m=+672.981925674"
Sep 11 13:59:44.990728 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:44.990683    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:44.991178 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:44.990744    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:45.003608 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.003577    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:45.454554 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.454518    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"]
Sep 11 13:59:45.460350 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.460312    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.464285 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.464259    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 13:59:45.469716 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.469688    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"]
Sep 11 13:59:45.501174 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.501130    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.501174 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.501175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.501395 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.501271    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.501395 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.501349    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.501395 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.501384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8jvs\" (UniqueName: \"kubernetes.io/projected/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-kube-api-access-m8jvs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.501498 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.501413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.601892 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.601845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.602095 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.601939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.602095 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.601966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8jvs\" (UniqueName: \"kubernetes.io/projected/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-kube-api-access-m8jvs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.602095 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.602004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.602095 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.602042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.602095 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.602075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.602396 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.602371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.602445 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.602413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.602484 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.602464    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.604662 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.604631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.604938 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.604920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.610513 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.610463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8jvs\" (UniqueName: \"kubernetes.io/projected/fcf4d975-a308-48b9-a8b4-e73c3c8a2a36-kube-api-access-m8jvs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4strr\" (UID: \"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:45.766685 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.766589    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-szh5n"
Sep 11 13:59:45.774579 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:45.774554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:46.056244 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:46.056185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"]
Sep 11 13:59:46.058865 ip-10-0-137-192 kubenswrapper[2563]: W0911 13:59:46.058831    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcf4d975_a308_48b9_a8b4_e73c3c8a2a36.slice/crio-5f5165f572050472b9ae47ef54c070beae4f8f69bfdcccc1058a02284a53f04f WatchSource:0}: Error finding container 5f5165f572050472b9ae47ef54c070beae4f8f69bfdcccc1058a02284a53f04f: Status 404 returned error can't find the container with id 5f5165f572050472b9ae47ef54c070beae4f8f69bfdcccc1058a02284a53f04f
Sep 11 13:59:46.761975 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:46.761939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr" event={"ID":"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36","Type":"ContainerStarted","Data":"50281ed2cfb43ac56f6072bd56e3a34683f5578fa6a58a3f410e6f1f1d9dd2e1"}
Sep 11 13:59:46.761975 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:46.761985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr" event={"ID":"fcf4d975-a308-48b9-a8b4-e73c3c8a2a36","Type":"ContainerStarted","Data":"5f5165f572050472b9ae47ef54c070beae4f8f69bfdcccc1058a02284a53f04f"}
Sep 11 13:59:46.780003 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:46.779947    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr" podStartSLOduration=1.57951387 podStartE2EDuration="1.77992812s" totalImagesPullingTime="200.41425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:46.777924562 +0000 UTC m=+683.036124586" watchObservedRunningTime="2026-09-11 13:59:46.77992812 +0000 UTC m=+683.038128129"
Sep 11 13:59:47.839975 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:47.839929    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 13:59:55.774991 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:55.774954    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:55.775410 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:55.775006    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:55.787914 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:55.787887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 13:59:55.823897 ip-10-0-137-192 kubenswrapper[2563]: I0911 13:59:55.823868    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4strr"
Sep 11 14:01:29.764496 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.764463    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"]
Sep 11 14:01:29.768202 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.768182    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.770593 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.770573    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-9c9fc1-openshift-default-dockercfg-wrqsm\""
Sep 11 14:01:29.780031 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.780003    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"]
Sep 11 14:01:29.843805 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.843768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-envoy\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.843993 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.843844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9594p\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-kube-api-access-9594p\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.843993 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.843893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-certs\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.843993 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.843935    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-podinfo\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.843993 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.843951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-token\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.843993 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.843972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3163d52a-3ce4-4250-88b0-e106e65c816b-istiod-ca-cert\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.844187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.844000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-credential-socket\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.844187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.844072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-socket\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.844187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.844116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-data\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945107 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9594p\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-kube-api-access-9594p\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945107 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-certs\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-podinfo\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-token\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3163d52a-3ce4-4250-88b0-e106e65c816b-istiod-ca-cert\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-credential-socket\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-socket\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-data\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-envoy\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945558    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-certs\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945678    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-credential-socket\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-socket\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.945896 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.945727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-data\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.946385 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.946361    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3163d52a-3ce4-4250-88b0-e106e65c816b-istiod-ca-cert\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.947835 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.947817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-envoy\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.948131 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.948114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-podinfo\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.954379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.954357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-token\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:29.954491 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:29.954432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9594p\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-kube-api-access-9594p\") pod \"e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") " pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:30.080118 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:30.080026    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:30.370168 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:30.370139    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"]
Sep 11 14:01:30.372493 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:01:30.372463    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3163d52a_3ce4_4250_88b0_e106e65c816b.slice/crio-9b6bf4675bf891b40fb67aeecfcc0ed6b10389379f9c348a34478b1c293300b6 WatchSource:0}: Error finding container 9b6bf4675bf891b40fb67aeecfcc0ed6b10389379f9c348a34478b1c293300b6: Status 404 returned error can't find the container with id 9b6bf4675bf891b40fb67aeecfcc0ed6b10389379f9c348a34478b1c293300b6
Sep 11 14:01:30.374633 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:30.374599    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:30.374779 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:30.374669    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:30.374779 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:30.374696    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:31.344125 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:31.344086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" event={"ID":"3163d52a-3ce4-4250-88b0-e106e65c816b","Type":"ContainerStarted","Data":"64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19"}
Sep 11 14:01:31.344544 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:31.344144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" event={"ID":"3163d52a-3ce4-4250-88b0-e106e65c816b","Type":"ContainerStarted","Data":"9b6bf4675bf891b40fb67aeecfcc0ed6b10389379f9c348a34478b1c293300b6"}
Sep 11 14:01:31.373829 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:31.373778    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podStartSLOduration=2.373760276 podStartE2EDuration="2.373760276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:31.371827272 +0000 UTC m=+787.630027281" watchObservedRunningTime="2026-09-11 14:01:31.373760276 +0000 UTC m=+787.631960276"
Sep 11 14:01:32.080957 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.080918    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:32.086920 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.086882    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:32.350488 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.350453    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:32.351376 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.351357    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:32.379403 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.379366    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"]
Sep 11 14:01:32.383537 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.383510    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.386095 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.386069    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-56cecb57-kserve-self-signed-certs\""
Sep 11 14:01:32.397298 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.397267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"]
Sep 11 14:01:32.468175 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.468129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-tmp-dir\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.468374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.468195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h8rl\" (UniqueName: \"kubernetes.io/projected/06caca19-cde9-4f27-baef-fddea3627cb5-kube-api-access-5h8rl\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.468374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.468277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-dshm\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.468374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.468320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06caca19-cde9-4f27-baef-fddea3627cb5-tls-certs\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.468374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.468355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-model-cache\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.468545 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.468481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-home\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.569739 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.569694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h8rl\" (UniqueName: \"kubernetes.io/projected/06caca19-cde9-4f27-baef-fddea3627cb5-kube-api-access-5h8rl\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.570018 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.569777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-dshm\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.570018 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.569822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06caca19-cde9-4f27-baef-fddea3627cb5-tls-certs\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.570018 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.569928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-model-cache\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.570205 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.570047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-home\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.570205 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.570095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-tmp-dir\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.570340 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.570318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-model-cache\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.570411 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.570383    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-home\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.570460 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.570445    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-tmp-dir\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.572310 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.572288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-dshm\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.572602 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.572583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06caca19-cde9-4f27-baef-fddea3627cb5-tls-certs\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.578291 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.578266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h8rl\" (UniqueName: \"kubernetes.io/projected/06caca19-cde9-4f27-baef-fddea3627cb5-kube-api-access-5h8rl\") pod \"e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") " pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.697710 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.697609    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:32.976742 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:32.976715    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"]
Sep 11 14:01:32.979434 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:01:32.979404    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06caca19_cde9_4f27_baef_fddea3627cb5.slice/crio-4d2c63ecb61fc0cda57a02f12edae16ef103e2f2188473fee95878e1b59b9115 WatchSource:0}: Error finding container 4d2c63ecb61fc0cda57a02f12edae16ef103e2f2188473fee95878e1b59b9115: Status 404 returned error can't find the container with id 4d2c63ecb61fc0cda57a02f12edae16ef103e2f2188473fee95878e1b59b9115
Sep 11 14:01:33.357155 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:33.357119    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6" event={"ID":"06caca19-cde9-4f27-baef-fddea3627cb5","Type":"ContainerStarted","Data":"7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796"}
Sep 11 14:01:33.357655 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:33.357167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6" event={"ID":"06caca19-cde9-4f27-baef-fddea3627cb5","Type":"ContainerStarted","Data":"4d2c63ecb61fc0cda57a02f12edae16ef103e2f2188473fee95878e1b59b9115"}
Sep 11 14:01:33.374845 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:33.374797    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6" podStartSLOduration=1.374738835 podStartE2EDuration="1.374738835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:33.373351218 +0000 UTC m=+789.631551226" watchObservedRunningTime="2026-09-11 14:01:33.374738835 +0000 UTC m=+789.632938844"
Sep 11 14:01:33.445255 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:33.445215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:01:37.953273 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:37.953239    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"]
Sep 11 14:01:37.958220 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:37.958195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:37.960838 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:37.960813    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-30fbcdfa-openshift-default-dockercfg-czzjk\""
Sep 11 14:01:37.968393 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:37.968363    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"]
Sep 11 14:01:38.022132 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-certs\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.022132 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-socket\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.022364 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-podinfo\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.022364 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-token\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.022364 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac12f43f-8148-4c91-a3fd-16b730506d69-istiod-ca-cert\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.022364 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-data\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.022364 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c77br\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-kube-api-access-c77br\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.022364 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-credential-socket\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.022580 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.022391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-envoy\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123634 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-podinfo\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123634 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-token\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac12f43f-8148-4c91-a3fd-16b730506d69-istiod-ca-cert\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-data\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c77br\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-kube-api-access-c77br\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-credential-socket\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-envoy\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-certs\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.123910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.123881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-socket\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.124551 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.124525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-socket\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.124653 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.124611    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-certs\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.124889 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.124865    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac12f43f-8148-4c91-a3fd-16b730506d69-istiod-ca-cert\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.124978 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.124959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-data\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.125227 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.125204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-credential-socket\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.126812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.126790    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-podinfo\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.127200 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.127176    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-envoy\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.132676 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.132647    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-token\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.133266 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.133243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c77br\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-kube-api-access-c77br\") pod \"e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") " pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.215389 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.215280    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"]
Sep 11 14:01:38.220851 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.220821    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.223481 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.223425    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-498d98-openshift-default-dockercfg-jzsjk\""
Sep 11 14:01:38.230852 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.230823    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"]
Sep 11 14:01:38.271451 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.271404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-credential-socket\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istiod-ca-cert\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-token\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-socket\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-data\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-podinfo\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-certs\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326427    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-envoy\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.327264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.326505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv4dr\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-kube-api-access-sv4dr\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427128 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-podinfo\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427308 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-certs\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427308 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-envoy\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427308 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv4dr\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-kube-api-access-sv4dr\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427308 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-credential-socket\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427536 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istiod-ca-cert\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427536 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-token\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427536 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-socket\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427690 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-data\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427690 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-certs\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427839 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-credential-socket\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.427966 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.427948    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-data\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.428148 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.428043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-socket\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.428148 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.428119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istiod-ca-cert\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.429886 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.429864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-envoy\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.430010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.429938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-podinfo\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.436622 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.436587    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-token\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.436732 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.436683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv4dr\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-kube-api-access-sv4dr\") pod \"e2e-autores-498d98-openshift-default-578d9d7866-gmktw\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") " pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.534797 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.534692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:38.559311 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.559284    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"]
Sep 11 14:01:38.561027 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:01:38.560994    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac12f43f_8148_4c91_a3fd_16b730506d69.slice/crio-fd04fb087c60c458bcf5190b2e653ebf4d151d6eb5cd63bc6d3526c3c0d93e99 WatchSource:0}: Error finding container fd04fb087c60c458bcf5190b2e653ebf4d151d6eb5cd63bc6d3526c3c0d93e99: Status 404 returned error can't find the container with id fd04fb087c60c458bcf5190b2e653ebf4d151d6eb5cd63bc6d3526c3c0d93e99
Sep 11 14:01:38.563317 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.563282    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:38.563427 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.563354    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:38.563427 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.563384    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:38.818651 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.818564    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"]
Sep 11 14:01:38.823590 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:01:38.823564    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c7cd44_40f5_4444_a7e2_9fd109c0faaa.slice/crio-5f2530dffc3b5365e0c68d8e172e1e2b77adc4602617ff35815c7b5cfea8b842 WatchSource:0}: Error finding container 5f2530dffc3b5365e0c68d8e172e1e2b77adc4602617ff35815c7b5cfea8b842: Status 404 returned error can't find the container with id 5f2530dffc3b5365e0c68d8e172e1e2b77adc4602617ff35815c7b5cfea8b842
Sep 11 14:01:38.826660 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.826615    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:38.826791 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.826735    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:38.826856 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:38.826805    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:39.401236 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:39.401204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" event={"ID":"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa","Type":"ContainerStarted","Data":"a953ecee4d0048d97b4575844d881a4486d8a60297cb87f7cf951463016e7847"}
Sep 11 14:01:39.401658 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:39.401271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" event={"ID":"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa","Type":"ContainerStarted","Data":"5f2530dffc3b5365e0c68d8e172e1e2b77adc4602617ff35815c7b5cfea8b842"}
Sep 11 14:01:39.403181 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:39.403146    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" event={"ID":"ac12f43f-8148-4c91-a3fd-16b730506d69","Type":"ContainerStarted","Data":"1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190"}
Sep 11 14:01:39.403344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:39.403197    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" event={"ID":"ac12f43f-8148-4c91-a3fd-16b730506d69","Type":"ContainerStarted","Data":"fd04fb087c60c458bcf5190b2e653ebf4d151d6eb5cd63bc6d3526c3c0d93e99"}
Sep 11 14:01:39.425994 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:39.425942    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" podStartSLOduration=1.425926434 podStartE2EDuration="1.425926434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:39.42246611 +0000 UTC m=+795.680666134" watchObservedRunningTime="2026-09-11 14:01:39.425926434 +0000 UTC m=+795.684126437"
Sep 11 14:01:39.443922 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:39.443860    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" podStartSLOduration=2.443838986 podStartE2EDuration="2.443838986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:39.443154667 +0000 UTC m=+795.701354687" watchObservedRunningTime="2026-09-11 14:01:39.443838986 +0000 UTC m=+795.702038996"
Sep 11 14:01:39.535961 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:39.535920    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:39.541857 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:39.541827    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:40.272174 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:40.272134    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:40.277377 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:40.277352    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:40.410668 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:40.410623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:40.411140 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:40.410843    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:40.411372 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:40.411354    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:01:40.411476 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:40.411460    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:01:42.698153 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:42.698107    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:42.698153 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:42.698167    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:42.737094 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:42.737061    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:43.444836 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:43.444808    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:01:45.735090 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:45.735002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:01:48.392991 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:48.392946    2563 patch_prober.go:28] interesting pod/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 14:01:48.393394 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:48.393027    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 14:01:51.484712 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:51.484680    2563 generic.go:356] "Generic (PLEG): container finished" podID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerID="64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19" exitCode=0
Sep 11 14:01:51.485181 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:51.484775    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" event={"ID":"3163d52a-3ce4-4250-88b0-e106e65c816b","Type":"ContainerDied","Data":"64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19"}
Sep 11 14:01:51.485228 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:51.485180    2563 scope.go:117] "RemoveContainer" containerID="64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19"
Sep 11 14:01:51.486283 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:51.486254    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:51.486398 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:51.486335    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:51.486398 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:51.486382    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:52.492480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:52.492448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" event={"ID":"3163d52a-3ce4-4250-88b0-e106e65c816b","Type":"ContainerStarted","Data":"24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad"}
Sep 11 14:01:53.080441 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:53.080399    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:53.080698 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:53.080674    2563 patch_prober.go:28] interesting pod/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 11 14:01:53.080820 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:53.080736    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 11 14:01:54.081399 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:54.081362    2563 patch_prober.go:28] interesting pod/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 11 14:01:54.081846 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:54.081427    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 11 14:01:55.066292 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:55.066249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:01:55.081093 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:55.081054    2563 patch_prober.go:28] interesting pod/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 11 14:01:55.081292 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:55.081114    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 11 14:01:56.081278 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:56.081163    2563 patch_prober.go:28] interesting pod/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 11 14:01:56.081278 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:56.081227    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 11 14:01:56.462303 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:56.461902    2563 patch_prober.go:28] interesting pod/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 14:01:56.462303 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:56.461979    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 14:01:57.080794 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:57.080714    2563 patch_prober.go:28] interesting pod/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 11 14:01:57.081126 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:57.081093    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 11 14:01:58.311197 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.311157    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:58.312895 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.312803    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:58.318339 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.318294    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:01:58.552783 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.552727    2563 generic.go:356] "Generic (PLEG): container finished" podID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerID="1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190" exitCode=0
Sep 11 14:01:58.552980 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.552784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" event={"ID":"ac12f43f-8148-4c91-a3fd-16b730506d69","Type":"ContainerDied","Data":"1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190"}
Sep 11 14:01:58.553345 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.553332    2563 scope.go:117] "RemoveContainer" containerID="1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190"
Sep 11 14:01:58.554477 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.554450    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:58.554536 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.554519    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:58.554572 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.554548    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:58.938832 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.934826    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"]
Sep 11 14:01:58.942521 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.942488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"]
Sep 11 14:01:58.942720 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.942617    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:58.945702 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:58.945671    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-da9534-openshift-default-dockercfg-c94h2\""
Sep 11 14:01:59.062099 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-certs\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.062284 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-podinfo\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.062284 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-token\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.062284 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76mcg\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-kube-api-access-76mcg\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.062395 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-credential-socket\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.062395 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istiod-ca-cert\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.062465 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-envoy\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.062500 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-data\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.062500 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.062482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-socket\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.163671 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.163629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-envoy\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.163671 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.163671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-data\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.163958 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.163699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-socket\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.163958 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.163841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-certs\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.163958 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.163904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-podinfo\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.163958 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.163939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-token\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.164163 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.163974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76mcg\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-kube-api-access-76mcg\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.164163 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.164035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-credential-socket\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.164163 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.164064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istiod-ca-cert\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.164163 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.164119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-data\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.164358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.164172    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-socket\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.164487 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.164466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-credential-socket\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.164561 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.164507    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-certs\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.164988 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.164957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istiod-ca-cert\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.166437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.166410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-envoy\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.166652 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.166630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-podinfo\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.182510 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.182478    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-token\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.183412 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.183388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76mcg\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-kube-api-access-76mcg\") pod \"e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") " pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.256878 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.256780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:01:59.562262 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.562171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" event={"ID":"ac12f43f-8148-4c91-a3fd-16b730506d69","Type":"ContainerStarted","Data":"2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3"}
Sep 11 14:01:59.580359 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.580327    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"]
Sep 11 14:01:59.582106 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:01:59.582069    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99724b07_84c6_4aac_a38f_7ff0e89c29aa.slice/crio-3fb1b270980c591e4d57c59ffeefbdde74edc883e0a9d09f9e243130d57b7358 WatchSource:0}: Error finding container 3fb1b270980c591e4d57c59ffeefbdde74edc883e0a9d09f9e243130d57b7358: Status 404 returned error can't find the container with id 3fb1b270980c591e4d57c59ffeefbdde74edc883e0a9d09f9e243130d57b7358
Sep 11 14:01:59.585464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.585426    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:59.585574 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.585516    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:01:59.585574 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:01:59.585543    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:00.272531 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:00.272497    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:02:00.272735 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:00.272603    2563 patch_prober.go:28] interesting pod/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 14:02:00.272735 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:00.272648    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 14:02:00.569623 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:00.569531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" event={"ID":"99724b07-84c6-4aac-a38f-7ff0e89c29aa","Type":"ContainerStarted","Data":"a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd"}
Sep 11 14:02:00.569623 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:00.569585    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" event={"ID":"99724b07-84c6-4aac-a38f-7ff0e89c29aa","Type":"ContainerStarted","Data":"3fb1b270980c591e4d57c59ffeefbdde74edc883e0a9d09f9e243130d57b7358"}
Sep 11 14:02:00.593971 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:00.593693    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" podStartSLOduration=2.5936698160000002 podStartE2EDuration="2.593669816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:00.592510529 +0000 UTC m=+816.850710545" watchObservedRunningTime="2026-09-11 14:02:00.593669816 +0000 UTC m=+816.851869827"
Sep 11 14:02:01.258707 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:01.258669    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:02:01.262539 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:01.262514    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:02:01.272269 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:01.272230    2563 patch_prober.go:28] interesting pod/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 14:02:01.272431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:01.272282    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 14:02:01.576065 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:01.575963    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:02:01.576966 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:01.576950    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:02:02.272683 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:02.272591    2563 patch_prober.go:28] interesting pod/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 14:02:02.272683 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:02.272654    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 14:02:03.272026 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:03.271983    2563 patch_prober.go:28] interesting pod/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 14:02:03.272592 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:03.272043    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 14:02:04.272400 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:04.272352    2563 patch_prober.go:28] interesting pod/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 14:02:04.272969 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:04.272413    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 14:02:05.354194 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:05.354160    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:02:05.354699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:05.354584    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:02:05.355864 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:05.355842    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:02:06.610080 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:06.610047    2563 generic.go:356] "Generic (PLEG): container finished" podID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerID="a953ecee4d0048d97b4575844d881a4486d8a60297cb87f7cf951463016e7847" exitCode=0
Sep 11 14:02:06.610605 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:06.610124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" event={"ID":"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa","Type":"ContainerDied","Data":"a953ecee4d0048d97b4575844d881a4486d8a60297cb87f7cf951463016e7847"}
Sep 11 14:02:06.610738 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:06.610721    2563 scope.go:117] "RemoveContainer" containerID="a953ecee4d0048d97b4575844d881a4486d8a60297cb87f7cf951463016e7847"
Sep 11 14:02:06.611802 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:06.611776    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:06.611902 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:06.611842    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:06.611902 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:06.611872    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:07.623823 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:07.623781    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" event={"ID":"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa","Type":"ContainerStarted","Data":"d1d484db47a48f4869a973ef0d9c744927c3a591a3ad07b193ee13480cb87e19"}
Sep 11 14:02:08.535084 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:08.535052    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:02:08.536183 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:08.536158    2563 patch_prober.go:28] interesting pod/e2e-autores-498d98-openshift-default-578d9d7866-gmktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 14:02:08.536307 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:08.536211    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 14:02:09.535228 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:09.535187    2563 patch_prober.go:28] interesting pod/e2e-autores-498d98-openshift-default-578d9d7866-gmktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 14:02:09.535719 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:09.535382    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 14:02:10.411205 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:10.411146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:02:10.535868 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:10.535830    2563 patch_prober.go:28] interesting pod/e2e-autores-498d98-openshift-default-578d9d7866-gmktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 14:02:10.536262 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:10.535906    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 14:02:11.535182 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:11.535148    2563 patch_prober.go:28] interesting pod/e2e-autores-498d98-openshift-default-578d9d7866-gmktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 14:02:11.535425 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:11.535403    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 14:02:12.348884 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:12.348853    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:02:12.536024 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:12.535981    2563 patch_prober.go:28] interesting pod/e2e-autores-498d98-openshift-default-578d9d7866-gmktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 14:02:12.536249 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:12.536046    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 14:02:13.410564 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:13.410533    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"]
Sep 11 14:02:13.411053 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:13.410769    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy" containerID="cri-o://2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3" gracePeriod=30
Sep 11 14:02:13.540901 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:13.540872    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:02:13.542066 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:13.542048    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:02:18.719659 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.719576    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:02:18.721431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.721398    2563 generic.go:356] "Generic (PLEG): container finished" podID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerID="2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3" exitCode=0
Sep 11 14:02:18.721978 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.721916    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" event={"ID":"ac12f43f-8148-4c91-a3fd-16b730506d69","Type":"ContainerDied","Data":"2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3"}
Sep 11 14:02:18.722234 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.722214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c" event={"ID":"ac12f43f-8148-4c91-a3fd-16b730506d69","Type":"ContainerDied","Data":"fd04fb087c60c458bcf5190b2e653ebf4d151d6eb5cd63bc6d3526c3c0d93e99"}
Sep 11 14:02:18.722396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.722382    2563 scope.go:117] "RemoveContainer" containerID="2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3"
Sep 11 14:02:18.740856 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.740821    2563 scope.go:117] "RemoveContainer" containerID="1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190"
Sep 11 14:02:18.752399 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.752373    2563 scope.go:117] "RemoveContainer" containerID="2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3"
Sep 11 14:02:18.752974 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:02:18.752952    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3\": container with ID starting with 2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3 not found: ID does not exist" containerID="2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3"
Sep 11 14:02:18.753122 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.753100    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3"} err="failed to get container status \"2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3\": rpc error: code = NotFound desc = could not find container \"2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3\": container with ID starting with 2aa965782ce8619d953bfe93898fb75849c07cb0b23e449608792f85b2a4fea3 not found: ID does not exist"
Sep 11 14:02:18.753220 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.753212    2563 scope.go:117] "RemoveContainer" containerID="1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190"
Sep 11 14:02:18.753625 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:02:18.753604    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190\": container with ID starting with 1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190 not found: ID does not exist" containerID="1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190"
Sep 11 14:02:18.753709 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.753632    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190"} err="failed to get container status \"1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190\": rpc error: code = NotFound desc = could not find container \"1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190\": container with ID starting with 1e3db0ed24bafc5d6d9f35f700dee394e22f73bbae641ddf807c8337ee697190 not found: ID does not exist"
Sep 11 14:02:18.779181 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-socket\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779219    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac12f43f-8148-4c91-a3fd-16b730506d69-istiod-ca-cert\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779256    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-certs\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779304    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c77br\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-kube-api-access-c77br\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779360    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-data\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779395    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-envoy\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779454    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-token\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779494    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:02:18.779610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779536    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-podinfo\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779587    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-credential-socket\") pod \"ac12f43f-8148-4c91-a3fd-16b730506d69\" (UID: \"ac12f43f-8148-4c91-a3fd-16b730506d69\") "
Sep 11 14:02:18.779610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779596    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:02:18.779972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac12f43f-8148-4c91-a3fd-16b730506d69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:02:18.779972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.779959    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:02:18.780079 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.780051    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:02:18.780308 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.780164    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:18.780308 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.780309    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:18.780455 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.780323    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac12f43f-8148-4c91-a3fd-16b730506d69-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:18.780455 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.780337    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:18.782486 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.782459    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:02:18.782486 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.782474    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:02:18.782623 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.782556    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:02:18.782930 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.782910    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-kube-api-access-c77br" (OuterVolumeSpecName: "kube-api-access-c77br") pod "ac12f43f-8148-4c91-a3fd-16b730506d69" (UID: "ac12f43f-8148-4c91-a3fd-16b730506d69"). InnerVolumeSpecName "kube-api-access-c77br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:02:18.881431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.881394    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:18.881431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.881422    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:18.881431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.881430    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:18.881431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.881438    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac12f43f-8148-4c91-a3fd-16b730506d69-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:18.881724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:18.881446    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c77br\" (UniqueName: \"kubernetes.io/projected/ac12f43f-8148-4c91-a3fd-16b730506d69-kube-api-access-c77br\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:19.728133 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:19.728101    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"
Sep 11 14:02:19.750804 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:19.750765    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"]
Sep 11 14:02:19.759373 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:19.759343    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-30fbcdfa-openshift-default-777f77ddb-hc66c"]
Sep 11 14:02:20.241887 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:20.241852    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" path="/var/lib/kubelet/pods/ac12f43f-8148-4c91-a3fd-16b730506d69/volumes"
Sep 11 14:02:23.755810 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:23.755778    2563 generic.go:356] "Generic (PLEG): container finished" podID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerID="a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd" exitCode=0
Sep 11 14:02:23.756245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:23.755839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" event={"ID":"99724b07-84c6-4aac-a38f-7ff0e89c29aa","Type":"ContainerDied","Data":"a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd"}
Sep 11 14:02:23.756289 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:23.756266    2563 scope.go:117] "RemoveContainer" containerID="a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd"
Sep 11 14:02:23.757484 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:23.757457    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:23.757542 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:23.757522    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:23.757582 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:23.757556    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:24.764260 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:24.764221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" event={"ID":"99724b07-84c6-4aac-a38f-7ff0e89c29aa","Type":"ContainerStarted","Data":"c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1"}
Sep 11 14:02:25.257227 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:25.257182    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:02:25.257508 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:25.257479    2563 patch_prober.go:28] interesting pod/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 11 14:02:25.257616 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:25.257551    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 11 14:02:26.257437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:26.257401    2563 patch_prober.go:28] interesting pod/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 11 14:02:26.257907 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:26.257469    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 11 14:02:27.258228 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:27.258191    2563 patch_prober.go:28] interesting pod/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 11 14:02:27.258638 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:27.258249    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 11 14:02:28.258100 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:28.258064    2563 patch_prober.go:28] interesting pod/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 11 14:02:28.258291 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:28.258130    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 11 14:02:29.257453 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:29.257415    2563 patch_prober.go:28] interesting pod/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 11 14:02:29.257671 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:29.257502    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 11 14:02:29.939905 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:29.939868    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:02:30.262195 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:30.262118    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:02:30.262501 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:30.262485    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:02:30.263294 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:30.263278    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:02:45.117698 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.117653    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"]
Sep 11 14:02:45.118215 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.118108    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy"
Sep 11 14:02:45.118215 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.118121    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy"
Sep 11 14:02:45.118215 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.118136    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy"
Sep 11 14:02:45.118215 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.118142    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy"
Sep 11 14:02:45.118347 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.118235    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy"
Sep 11 14:02:45.118347 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.118244    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac12f43f-8148-4c91-a3fd-16b730506d69" containerName="istio-proxy"
Sep 11 14:02:45.121222 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.121206    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"
Sep 11 14:02:45.125171 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.125145    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:02:45.125318 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.125188    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:02:45.131412 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.131384    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"]
Sep 11 14:02:45.246315 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.246283    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"]
Sep 11 14:02:45.249111 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.249094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.251491 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.251466    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-56cecb57-kserve-self-signed-certs\""
Sep 11 14:02:45.258135 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.258110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3c2700-713c-4eb9-b3a0-7a8b9c174c23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts\" (UID: \"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"
Sep 11 14:02:45.259348 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.259326    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"]
Sep 11 14:02:45.359064 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.359034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/391fc17e-7feb-4aa2-8dda-02abd78128a9-tls-certs\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.359253 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.359094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-model-cache\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.359253 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.359180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-dshm\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.359336 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.359263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-home\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.359336 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.359292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-tmp-dir\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.359432 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.359341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3c2700-713c-4eb9-b3a0-7a8b9c174c23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts\" (UID: \"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"
Sep 11 14:02:45.359432 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.359376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5mdq\" (UniqueName: \"kubernetes.io/projected/391fc17e-7feb-4aa2-8dda-02abd78128a9-kube-api-access-n5mdq\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.360009 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.359986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3c2700-713c-4eb9-b3a0-7a8b9c174c23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts\" (UID: \"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"
Sep 11 14:02:45.432888 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.432718    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"
Sep 11 14:02:45.460079 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.460048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/391fc17e-7feb-4aa2-8dda-02abd78128a9-tls-certs\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.460244 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.460177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-model-cache\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.460244 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.460235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-dshm\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.460371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.460308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-home\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.460371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.460345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-tmp-dir\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.460478 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.460426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5mdq\" (UniqueName: \"kubernetes.io/projected/391fc17e-7feb-4aa2-8dda-02abd78128a9-kube-api-access-n5mdq\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.461106 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.461076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-home\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.461299 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.461168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-model-cache\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.461441 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.461416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-tmp-dir\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.462701 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.462682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-dshm\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.463136 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.463116    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/391fc17e-7feb-4aa2-8dda-02abd78128a9-tls-certs\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.469074 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.469047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5mdq\" (UniqueName: \"kubernetes.io/projected/391fc17e-7feb-4aa2-8dda-02abd78128a9-kube-api-access-n5mdq\") pod \"e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") " pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.558553 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.558514    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:45.717488 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.717140    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"]
Sep 11 14:02:45.721874 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:02:45.721738    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3c2700_713c_4eb9_b3a0_7a8b9c174c23.slice/crio-e9cff547016d497a6937785daf981c0119ca13ccb8b1cd4df479dcd9af80de56 WatchSource:0}: Error finding container e9cff547016d497a6937785daf981c0119ca13ccb8b1cd4df479dcd9af80de56: Status 404 returned error can't find the container with id e9cff547016d497a6937785daf981c0119ca13ccb8b1cd4df479dcd9af80de56
Sep 11 14:02:45.834549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.834522    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"]
Sep 11 14:02:45.836467 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:02:45.836435    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod391fc17e_7feb_4aa2_8dda_02abd78128a9.slice/crio-3a4e1301ea69bcec7fe6f42994049b4824d24010b56e2845f192ce8b2e6df6f5 WatchSource:0}: Error finding container 3a4e1301ea69bcec7fe6f42994049b4824d24010b56e2845f192ce8b2e6df6f5: Status 404 returned error can't find the container with id 3a4e1301ea69bcec7fe6f42994049b4824d24010b56e2845f192ce8b2e6df6f5
Sep 11 14:02:45.896470 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.896438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts" event={"ID":"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23","Type":"ContainerStarted","Data":"e9cff547016d497a6937785daf981c0119ca13ccb8b1cd4df479dcd9af80de56"}
Sep 11 14:02:45.898267 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:45.898235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm" event={"ID":"391fc17e-7feb-4aa2-8dda-02abd78128a9","Type":"ContainerStarted","Data":"3a4e1301ea69bcec7fe6f42994049b4824d24010b56e2845f192ce8b2e6df6f5"}
Sep 11 14:02:46.906431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:46.906379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm" event={"ID":"391fc17e-7feb-4aa2-8dda-02abd78128a9","Type":"ContainerStarted","Data":"04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d"}
Sep 11 14:02:46.923994 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:46.923941    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm" podStartSLOduration=1.9239236 podStartE2EDuration="1.9239236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:46.92255179 +0000 UTC m=+863.180751798" watchObservedRunningTime="2026-09-11 14:02:46.9239236 +0000 UTC m=+863.182123634"
Sep 11 14:02:48.921784 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:48.921736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts" event={"ID":"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23","Type":"ContainerStarted","Data":"9592f965fe5898043d74c6e6b803faae295b8a260b86e470f5bb990a6f05d054"}
Sep 11 14:02:48.937950 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:48.937903    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts" podStartSLOduration=1.214879681 podStartE2EDuration="3.937886939s" totalImagesPullingTime="2.723007258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:48.936039554 +0000 UTC m=+865.194239561" watchObservedRunningTime="2026-09-11 14:02:48.937886939 +0000 UTC m=+865.196087002"
Sep 11 14:02:49.738984 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:49.738945    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:02:49.928932 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:49.928896    2563 generic.go:356] "Generic (PLEG): container finished" podID="ce3c2700-713c-4eb9-b3a0-7a8b9c174c23" containerID="9592f965fe5898043d74c6e6b803faae295b8a260b86e470f5bb990a6f05d054" exitCode=0
Sep 11 14:02:49.929426 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:49.928972    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts" event={"ID":"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23","Type":"ContainerDied","Data":"9592f965fe5898043d74c6e6b803faae295b8a260b86e470f5bb990a6f05d054"}
Sep 11 14:02:51.081926 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:51.081897    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"
Sep 11 14:02:51.119835 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:51.119795    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3c2700-713c-4eb9-b3a0-7a8b9c174c23-maas-api-service-ca\") pod \"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23\" (UID: \"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23\") "
Sep 11 14:02:51.120194 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:51.120171    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce3c2700-713c-4eb9-b3a0-7a8b9c174c23-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ce3c2700-713c-4eb9-b3a0-7a8b9c174c23" (UID: "ce3c2700-713c-4eb9-b3a0-7a8b9c174c23"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:02:51.221052 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:51.220965    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3c2700-713c-4eb9-b3a0-7a8b9c174c23-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:02:51.944809 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:51.944772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"
Sep 11 14:02:51.945039 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:51.944800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts" event={"ID":"ce3c2700-713c-4eb9-b3a0-7a8b9c174c23","Type":"ContainerDied","Data":"e9cff547016d497a6937785daf981c0119ca13ccb8b1cd4df479dcd9af80de56"}
Sep 11 14:02:51.945039 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:51.944901    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9cff547016d497a6937785daf981c0119ca13ccb8b1cd4df479dcd9af80de56"
Sep 11 14:02:55.559040 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:55.558986    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:55.559040 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:55.559057    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:55.572291 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:55.572262    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:02:55.979524 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:02:55.979493    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:03:02.329407 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:02.329362    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"]
Sep 11 14:03:02.332364 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:02.332334    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0d4b8af1-nx2ts"]
Sep 11 14:03:03.174746 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.174702    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"]
Sep 11 14:03:03.175606 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.175581    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce3c2700-713c-4eb9-b3a0-7a8b9c174c23" containerName="revoke-keys"
Sep 11 14:03:03.175606 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.175607    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce3c2700-713c-4eb9-b3a0-7a8b9c174c23" containerName="revoke-keys"
Sep 11 14:03:03.175869 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.175854    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce3c2700-713c-4eb9-b3a0-7a8b9c174c23" containerName="revoke-keys"
Sep 11 14:03:03.180195 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.180172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.182903 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.182880    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bd7f15a9-openshift-default-dockercfg-vbc7l\""
Sep 11 14:03:03.189568 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.189541    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"]
Sep 11 14:03:03.247857 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.247814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-token\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.248060 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.247882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-certs\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.248060 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.247910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-data\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.248060 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.247935    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-envoy\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.248060 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.248014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-socket\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.248248 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.248090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-podinfo\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.248248 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.248117    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-credential-socket\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.248248 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.248140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istiod-ca-cert\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.248248 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.248234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8gtq\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-kube-api-access-l8gtq\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.349543 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.349501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-certs\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.349543 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.349544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-data\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.349566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-envoy\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.349703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-socket\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.349835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-podinfo\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.349865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-credential-socket\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.349900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istiod-ca-cert\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.349965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8gtq\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-kube-api-access-l8gtq\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.350019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-token\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.350071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-certs\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350542 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.350242    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-socket\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350542 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.350287    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-data\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.350542 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.350313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-credential-socket\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.351103 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.351078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istiod-ca-cert\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.352237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.352213    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-envoy\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.352476 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.352459    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-podinfo\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.359245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.359217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-token\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.359356 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.359338    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8gtq\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-kube-api-access-l8gtq\") pod \"e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") " pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.491418 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.491316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:03.774331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.774302    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"]
Sep 11 14:03:03.775863 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:03:03.775830    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1443bee_1c53_4e1b_b96d_ad6bb2822f83.slice/crio-0352548151f95940014b56d4f77dce0f685215dbc8d4363619dcaa44aed4b30b WatchSource:0}: Error finding container 0352548151f95940014b56d4f77dce0f685215dbc8d4363619dcaa44aed4b30b: Status 404 returned error can't find the container with id 0352548151f95940014b56d4f77dce0f685215dbc8d4363619dcaa44aed4b30b
Sep 11 14:03:03.778074 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.778046    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:03.778136 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.778114    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:03.778179 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:03.778142    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:04.019020 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:04.018976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" event={"ID":"e1443bee-1c53-4e1b-b96d-ad6bb2822f83","Type":"ContainerStarted","Data":"ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d"}
Sep 11 14:03:04.019231 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:04.019052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" event={"ID":"e1443bee-1c53-4e1b-b96d-ad6bb2822f83","Type":"ContainerStarted","Data":"0352548151f95940014b56d4f77dce0f685215dbc8d4363619dcaa44aed4b30b"}
Sep 11 14:03:04.042000 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:04.041887    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podStartSLOduration=1.041866873 podStartE2EDuration="1.041866873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:04.03917375 +0000 UTC m=+880.297373759" watchObservedRunningTime="2026-09-11 14:03:04.041866873 +0000 UTC m=+880.300066882"
Sep 11 14:03:04.241265 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:04.241230    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce3c2700-713c-4eb9-b3a0-7a8b9c174c23" path="/var/lib/kubelet/pods/ce3c2700-713c-4eb9-b3a0-7a8b9c174c23/volumes"
Sep 11 14:03:04.491470 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:04.491439    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:04.493471 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:04.493445    2563 patch_prober.go:28] interesting pod/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 14:03:04.493643 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:04.493494    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 14:03:05.496333 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:05.496298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:06.030097 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:06.030070    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:06.031073 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:06.031050    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:07.241512 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:07.241478    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:03:19.164691 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.164601    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"]
Sep 11 14:03:19.168461 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.168432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.172323 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.172292    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9c9fc1\"/\"default-dockercfg-524ff\""
Sep 11 14:03:19.172323 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.172320    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9c9fc1\"/\"kube-root-ca.crt\""
Sep 11 14:03:19.172562 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.172330    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9c9fc1\"/\"openshift-service-ca.crt\""
Sep 11 14:03:19.172562 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.172292    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9c9fc1\"/\"test-model-9c9fc1-kserve-self-signed-certs\""
Sep 11 14:03:19.178181 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.178153    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"]
Sep 11 14:03:19.325916 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.325874    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-home\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.325916 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.325922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-model-cache\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.326187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.325957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gshdp\" (UniqueName: \"kubernetes.io/projected/d544fc62-df68-4c44-8357-b17f69958b75-kube-api-access-gshdp\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.326187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.326053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-dshm\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.326187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.326090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-tmp-dir\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.326187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.326125    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d544fc62-df68-4c44-8357-b17f69958b75-tls-certs\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.428007 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.427904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-home\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.428821 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.428796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-model-cache\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.429369 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.429349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gshdp\" (UniqueName: \"kubernetes.io/projected/d544fc62-df68-4c44-8357-b17f69958b75-kube-api-access-gshdp\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.429586 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.429561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-dshm\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.429711 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.428669    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-home\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.429836 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.429703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-tmp-dir\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.429976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.429960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d544fc62-df68-4c44-8357-b17f69958b75-tls-certs\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.430396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.430363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-tmp-dir\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.430493 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.429251    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-model-cache\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.433078 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.433033    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-dshm\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.433697 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.433673    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d544fc62-df68-4c44-8357-b17f69958b75-tls-certs\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.439737 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.439680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gshdp\" (UniqueName: \"kubernetes.io/projected/d544fc62-df68-4c44-8357-b17f69958b75-kube-api-access-gshdp\") pod \"test-model-9c9fc1-kserve-6dfc56f84c-fftgt\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") " pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.510447 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.510406    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:19.792594 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:19.792564    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"]
Sep 11 14:03:19.795307 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:03:19.795274    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd544fc62_df68_4c44_8357_b17f69958b75.slice/crio-acaf797231bd823d6491d7c471fb1ee3585465f20f164d61b7986b6b282d8f81 WatchSource:0}: Error finding container acaf797231bd823d6491d7c471fb1ee3585465f20f164d61b7986b6b282d8f81: Status 404 returned error can't find the container with id acaf797231bd823d6491d7c471fb1ee3585465f20f164d61b7986b6b282d8f81
Sep 11 14:03:20.135637 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:20.135583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt" event={"ID":"d544fc62-df68-4c44-8357-b17f69958b75","Type":"ContainerStarted","Data":"b699e3718cf52f9e8dfa6e4bd591ecc92b85df9da22c274043edb17dfe6a68db"}
Sep 11 14:03:20.135877 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:20.135651    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt" event={"ID":"d544fc62-df68-4c44-8357-b17f69958b75","Type":"ContainerStarted","Data":"acaf797231bd823d6491d7c471fb1ee3585465f20f164d61b7986b6b282d8f81"}
Sep 11 14:03:20.156230 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:20.156168    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt" podStartSLOduration=1.156149219 podStartE2EDuration="1.156149219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:20.154127918 +0000 UTC m=+896.412327926" watchObservedRunningTime="2026-09-11 14:03:20.156149219 +0000 UTC m=+896.414349227"
Sep 11 14:03:21.456744 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:21.456703    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"]
Sep 11 14:03:24.282874 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.282845    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:03:24.285350 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.285317    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:03:24.861995 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.861955    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"]
Sep 11 14:03:24.870601 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.870574    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"
Sep 11 14:03:24.872098 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.872068    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"]
Sep 11 14:03:24.873312 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.873287    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:03:24.873422 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.873287    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:03:24.886271 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.886237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96e2924-9623-4a3a-b87f-e97ad8c66a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-498d98-nznhq\" (UID: \"e96e2924-9623-4a3a-b87f-e97ad8c66a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"
Sep 11 14:03:24.987837 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.987802    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96e2924-9623-4a3a-b87f-e97ad8c66a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-498d98-nznhq\" (UID: \"e96e2924-9623-4a3a-b87f-e97ad8c66a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"
Sep 11 14:03:24.988500 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:24.988469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96e2924-9623-4a3a-b87f-e97ad8c66a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-498d98-nznhq\" (UID: \"e96e2924-9623-4a3a-b87f-e97ad8c66a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"
Sep 11 14:03:25.183683 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:25.183596    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"
Sep 11 14:03:25.475818 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:25.475780    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"]
Sep 11 14:03:25.477204 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:03:25.477176    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96e2924_9623_4a3a_b87f_e97ad8c66a60.slice/crio-15263754756edccb55c0eadeb9d42bc09a6ad170bb1a3867fee5d09c33b730f5 WatchSource:0}: Error finding container 15263754756edccb55c0eadeb9d42bc09a6ad170bb1a3867fee5d09c33b730f5: Status 404 returned error can't find the container with id 15263754756edccb55c0eadeb9d42bc09a6ad170bb1a3867fee5d09c33b730f5
Sep 11 14:03:26.174326 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:26.174285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq" event={"ID":"e96e2924-9623-4a3a-b87f-e97ad8c66a60","Type":"ContainerStarted","Data":"c3e2baae9b624a9bc3daed1a02baaf7edea2c0f0d36248ede946e6396a41acff"}
Sep 11 14:03:26.174542 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:26.174346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq" event={"ID":"e96e2924-9623-4a3a-b87f-e97ad8c66a60","Type":"ContainerStarted","Data":"15263754756edccb55c0eadeb9d42bc09a6ad170bb1a3867fee5d09c33b730f5"}
Sep 11 14:03:26.191166 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:26.191111    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq" podStartSLOduration=2.19109386 podStartE2EDuration="2.19109386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:26.188485209 +0000 UTC m=+902.446685218" watchObservedRunningTime="2026-09-11 14:03:26.19109386 +0000 UTC m=+902.449293870"
Sep 11 14:03:27.183538 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:27.183498    2563 generic.go:356] "Generic (PLEG): container finished" podID="e96e2924-9623-4a3a-b87f-e97ad8c66a60" containerID="c3e2baae9b624a9bc3daed1a02baaf7edea2c0f0d36248ede946e6396a41acff" exitCode=0
Sep 11 14:03:27.183976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:27.183587    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq" event={"ID":"e96e2924-9623-4a3a-b87f-e97ad8c66a60","Type":"ContainerDied","Data":"c3e2baae9b624a9bc3daed1a02baaf7edea2c0f0d36248ede946e6396a41acff"}
Sep 11 14:03:27.233870 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:27.233824    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:03:28.366695 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:28.366666    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"
Sep 11 14:03:28.426204 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:28.426167    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96e2924-9623-4a3a-b87f-e97ad8c66a60-maas-api-service-ca\") pod \"e96e2924-9623-4a3a-b87f-e97ad8c66a60\" (UID: \"e96e2924-9623-4a3a-b87f-e97ad8c66a60\") "
Sep 11 14:03:28.426644 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:28.426615    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e96e2924-9623-4a3a-b87f-e97ad8c66a60-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e96e2924-9623-4a3a-b87f-e97ad8c66a60" (UID: "e96e2924-9623-4a3a-b87f-e97ad8c66a60"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:03:28.528408 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:28.528284    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96e2924-9623-4a3a-b87f-e97ad8c66a60-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:29.198725 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.198690    2563 generic.go:356] "Generic (PLEG): container finished" podID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerID="ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d" exitCode=0
Sep 11 14:03:29.198943 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.198779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" event={"ID":"e1443bee-1c53-4e1b-b96d-ad6bb2822f83","Type":"ContainerDied","Data":"ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d"}
Sep 11 14:03:29.199354 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.199330    2563 scope.go:117] "RemoveContainer" containerID="ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d"
Sep 11 14:03:29.200248 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.200223    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"
Sep 11 14:03:29.200305 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.200267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq" event={"ID":"e96e2924-9623-4a3a-b87f-e97ad8c66a60","Type":"ContainerDied","Data":"15263754756edccb55c0eadeb9d42bc09a6ad170bb1a3867fee5d09c33b730f5"}
Sep 11 14:03:29.200360 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.200314    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15263754756edccb55c0eadeb9d42bc09a6ad170bb1a3867fee5d09c33b730f5"
Sep 11 14:03:29.200576 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.200544    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:29.200716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.200626    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:29.200716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.200683    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:29.510953 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.510849    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:29.510953 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.510900    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:29.524162 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:29.524130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:30.218036 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:30.217983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" event={"ID":"e1443bee-1c53-4e1b-b96d-ad6bb2822f83","Type":"ContainerStarted","Data":"97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397"}
Sep 11 14:03:30.241681 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:30.241641    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:03:30.491735 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:30.491639    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:30.491953 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:30.491932    2563 patch_prober.go:28] interesting pod/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 14:03:30.492006 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:30.491982    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 14:03:31.047886 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.047844    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"]
Sep 11 14:03:31.048459 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.048440    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e96e2924-9623-4a3a-b87f-e97ad8c66a60" containerName="revoke-keys"
Sep 11 14:03:31.048505 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.048462    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e96e2924-9623-4a3a-b87f-e97ad8c66a60" containerName="revoke-keys"
Sep 11 14:03:31.048562 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.048553    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e96e2924-9623-4a3a-b87f-e97ad8c66a60" containerName="revoke-keys"
Sep 11 14:03:31.051126 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.051103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"
Sep 11 14:03:31.053641 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.053621    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:03:31.053918 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.053897    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:03:31.060984 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.060956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"]
Sep 11 14:03:31.155814 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.155731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3359d3-f4df-4587-95d5-53cd3049bb17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n\" (UID: \"8f3359d3-f4df-4587-95d5-53cd3049bb17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"
Sep 11 14:03:31.257355 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.257321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3359d3-f4df-4587-95d5-53cd3049bb17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n\" (UID: \"8f3359d3-f4df-4587-95d5-53cd3049bb17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"
Sep 11 14:03:31.258292 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.258269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3359d3-f4df-4587-95d5-53cd3049bb17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n\" (UID: \"8f3359d3-f4df-4587-95d5-53cd3049bb17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"
Sep 11 14:03:31.364599 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.364568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"
Sep 11 14:03:31.492475 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.492432    2563 patch_prober.go:28] interesting pod/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 14:03:31.492646 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.492523    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 14:03:31.665410 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:31.665375    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"]
Sep 11 14:03:31.666080 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:03:31.666055    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3359d3_f4df_4587_95d5_53cd3049bb17.slice/crio-381e5c928468ea98167b936b2bb6fe9d3f5b2c2623d3f4ff847d7a07dcd56976 WatchSource:0}: Error finding container 381e5c928468ea98167b936b2bb6fe9d3f5b2c2623d3f4ff847d7a07dcd56976: Status 404 returned error can't find the container with id 381e5c928468ea98167b936b2bb6fe9d3f5b2c2623d3f4ff847d7a07dcd56976
Sep 11 14:03:32.246245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:32.246151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n" event={"ID":"8f3359d3-f4df-4587-95d5-53cd3049bb17","Type":"ContainerStarted","Data":"d1612ee5709797af03fdf6198f785bdf2bf52826e4ea572d2f5b1742253c83d8"}
Sep 11 14:03:32.246245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:32.246208    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n" event={"ID":"8f3359d3-f4df-4587-95d5-53cd3049bb17","Type":"ContainerStarted","Data":"381e5c928468ea98167b936b2bb6fe9d3f5b2c2623d3f4ff847d7a07dcd56976"}
Sep 11 14:03:32.254897 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:32.254836    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n" podStartSLOduration=1.25481938 podStartE2EDuration="1.25481938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:32.251782832 +0000 UTC m=+908.509982838" watchObservedRunningTime="2026-09-11 14:03:32.25481938 +0000 UTC m=+908.513019388"
Sep 11 14:03:32.492272 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:32.492237    2563 patch_prober.go:28] interesting pod/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 14:03:32.492438 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:32.492295    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 14:03:33.244888 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:33.244848    2563 generic.go:356] "Generic (PLEG): container finished" podID="8f3359d3-f4df-4587-95d5-53cd3049bb17" containerID="d1612ee5709797af03fdf6198f785bdf2bf52826e4ea572d2f5b1742253c83d8" exitCode=0
Sep 11 14:03:33.245109 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:33.244944    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n" event={"ID":"8f3359d3-f4df-4587-95d5-53cd3049bb17","Type":"ContainerDied","Data":"d1612ee5709797af03fdf6198f785bdf2bf52826e4ea572d2f5b1742253c83d8"}
Sep 11 14:03:33.492733 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:33.492695    2563 patch_prober.go:28] interesting pod/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 14:03:33.493174 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:33.492788    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 14:03:34.434411 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:34.434381    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"
Sep 11 14:03:34.488633 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:34.488596    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3359d3-f4df-4587-95d5-53cd3049bb17-maas-api-service-ca\") pod \"8f3359d3-f4df-4587-95d5-53cd3049bb17\" (UID: \"8f3359d3-f4df-4587-95d5-53cd3049bb17\") "
Sep 11 14:03:34.489037 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:34.489012    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f3359d3-f4df-4587-95d5-53cd3049bb17-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f3359d3-f4df-4587-95d5-53cd3049bb17" (UID: "8f3359d3-f4df-4587-95d5-53cd3049bb17"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:03:34.492136 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:34.492111    2563 patch_prober.go:28] interesting pod/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 14:03:34.492279 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:34.492163    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 14:03:34.589541 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:34.589451    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3359d3-f4df-4587-95d5-53cd3049bb17-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:35.258999 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:35.258970    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"
Sep 11 14:03:35.258999 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:35.258983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n" event={"ID":"8f3359d3-f4df-4587-95d5-53cd3049bb17","Type":"ContainerDied","Data":"381e5c928468ea98167b936b2bb6fe9d3f5b2c2623d3f4ff847d7a07dcd56976"}
Sep 11 14:03:35.259246 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:35.259026    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="381e5c928468ea98167b936b2bb6fe9d3f5b2c2623d3f4ff847d7a07dcd56976"
Sep 11 14:03:35.492477 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:35.492439    2563 patch_prober.go:28] interesting pod/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 14:03:35.492706 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:35.492497    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 14:03:35.883714 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:35.883671    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"]
Sep 11 14:03:35.886191 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:35.886167    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-498d98-nznhq"]
Sep 11 14:03:36.030429 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.030388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:36.241465 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.241368    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e96e2924-9623-4a3a-b87f-e97ad8c66a60" path="/var/lib/kubelet/pods/e96e2924-9623-4a3a-b87f-e97ad8c66a60/volumes"
Sep 11 14:03:36.477856 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.477803    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm" podUID="391fc17e-7feb-4aa2-8dda-02abd78128a9" containerName="main" containerID="cri-o://04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d" gracePeriod=45
Sep 11 14:03:36.496352 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.496330    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:36.497402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.497384    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:36.743465 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.742703    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:03:36.813047 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-model-cache\") pod \"391fc17e-7feb-4aa2-8dda-02abd78128a9\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") "
Sep 11 14:03:36.813240 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813067    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-dshm\") pod \"391fc17e-7feb-4aa2-8dda-02abd78128a9\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") "
Sep 11 14:03:36.813240 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813095    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-tmp-dir\") pod \"391fc17e-7feb-4aa2-8dda-02abd78128a9\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") "
Sep 11 14:03:36.813240 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813151    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/391fc17e-7feb-4aa2-8dda-02abd78128a9-tls-certs\") pod \"391fc17e-7feb-4aa2-8dda-02abd78128a9\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") "
Sep 11 14:03:36.813240 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813173    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-home\") pod \"391fc17e-7feb-4aa2-8dda-02abd78128a9\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") "
Sep 11 14:03:36.813240 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813232    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5mdq\" (UniqueName: \"kubernetes.io/projected/391fc17e-7feb-4aa2-8dda-02abd78128a9-kube-api-access-n5mdq\") pod \"391fc17e-7feb-4aa2-8dda-02abd78128a9\" (UID: \"391fc17e-7feb-4aa2-8dda-02abd78128a9\") "
Sep 11 14:03:36.813511 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813348    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "391fc17e-7feb-4aa2-8dda-02abd78128a9" (UID: "391fc17e-7feb-4aa2-8dda-02abd78128a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:36.813511 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813376    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "391fc17e-7feb-4aa2-8dda-02abd78128a9" (UID: "391fc17e-7feb-4aa2-8dda-02abd78128a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:36.813619 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813509    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-home" (OuterVolumeSpecName: "home") pod "391fc17e-7feb-4aa2-8dda-02abd78128a9" (UID: "391fc17e-7feb-4aa2-8dda-02abd78128a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:36.813658 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813638    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.813701 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813657    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.813701 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.813667    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.815642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.815614    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-dshm" (OuterVolumeSpecName: "dshm") pod "391fc17e-7feb-4aa2-8dda-02abd78128a9" (UID: "391fc17e-7feb-4aa2-8dda-02abd78128a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:36.815725 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.815620    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/391fc17e-7feb-4aa2-8dda-02abd78128a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "391fc17e-7feb-4aa2-8dda-02abd78128a9" (UID: "391fc17e-7feb-4aa2-8dda-02abd78128a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:03:36.815725 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.815691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/391fc17e-7feb-4aa2-8dda-02abd78128a9-kube-api-access-n5mdq" (OuterVolumeSpecName: "kube-api-access-n5mdq") pod "391fc17e-7feb-4aa2-8dda-02abd78128a9" (UID: "391fc17e-7feb-4aa2-8dda-02abd78128a9"). InnerVolumeSpecName "kube-api-access-n5mdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:36.915213 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.915181    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/391fc17e-7feb-4aa2-8dda-02abd78128a9-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.915213 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.915209    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5mdq\" (UniqueName: \"kubernetes.io/projected/391fc17e-7feb-4aa2-8dda-02abd78128a9-kube-api-access-n5mdq\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.915213 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:36.915220    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/391fc17e-7feb-4aa2-8dda-02abd78128a9-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:37.272532 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.272499    2563 generic.go:356] "Generic (PLEG): container finished" podID="391fc17e-7feb-4aa2-8dda-02abd78128a9" containerID="04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d" exitCode=0
Sep 11 14:03:37.272726 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.272543    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"
Sep 11 14:03:37.272726 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.272586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm" event={"ID":"391fc17e-7feb-4aa2-8dda-02abd78128a9","Type":"ContainerDied","Data":"04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d"}
Sep 11 14:03:37.272726 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.272630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm" event={"ID":"391fc17e-7feb-4aa2-8dda-02abd78128a9","Type":"ContainerDied","Data":"3a4e1301ea69bcec7fe6f42994049b4824d24010b56e2845f192ce8b2e6df6f5"}
Sep 11 14:03:37.272726 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.272650    2563 scope.go:117] "RemoveContainer" containerID="04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d"
Sep 11 14:03:37.292324 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.292292    2563 scope.go:117] "RemoveContainer" containerID="04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d"
Sep 11 14:03:37.292865 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:03:37.292819    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d\": container with ID starting with 04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d not found: ID does not exist" containerID="04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d"
Sep 11 14:03:37.293007 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.292865    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d"} err="failed to get container status \"04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d\": rpc error: code = NotFound desc = could not find container \"04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d\": container with ID starting with 04045a5724e4432cdcf301b14f7e873dc0320153461c3295ccac4e303eaa5e1d not found: ID does not exist"
Sep 11 14:03:37.303878 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.303828    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"]
Sep 11 14:03:37.315237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.315202    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-56cecb57-kserve-687fdf7cb9-7pjzm"]
Sep 11 14:03:37.572410 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.572316    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"]
Sep 11 14:03:37.577390 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:37.577361    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd7f15a9-kxp7n"]
Sep 11 14:03:38.240716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:38.240679    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="391fc17e-7feb-4aa2-8dda-02abd78128a9" path="/var/lib/kubelet/pods/391fc17e-7feb-4aa2-8dda-02abd78128a9/volumes"
Sep 11 14:03:38.241125 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:38.241057    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f3359d3-f4df-4587-95d5-53cd3049bb17" path="/var/lib/kubelet/pods/8f3359d3-f4df-4587-95d5-53cd3049bb17/volumes"
Sep 11 14:03:38.640480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:38.640451    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"]
Sep 11 14:03:38.640688 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:38.640664    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy" containerID="cri-o://97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397" gracePeriod=30
Sep 11 14:03:43.173310 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.173273    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"]
Sep 11 14:03:43.173823 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.173509    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy" containerID="cri-o://d1d484db47a48f4869a973ef0d9c744927c3a591a3ad07b193ee13480cb87e19" gracePeriod=30
Sep 11 14:03:43.544113 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.544021    2563 patch_prober.go:28] interesting pod/e2e-autores-498d98-openshift-default-578d9d7866-gmktw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:03:43.544113 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.544087    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:03:43.907723 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.907692    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:43.990263 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990229    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-token\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.990437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990277    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-data\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.990437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990301    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8gtq\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-kube-api-access-l8gtq\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.990437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990332    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-envoy\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.990437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-certs\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.990437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990420    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-credential-socket\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.990730 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990467    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-podinfo\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.990730 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990545    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-socket\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.990730 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.990613    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istiod-ca-cert\") pod \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\" (UID: \"e1443bee-1c53-4e1b-b96d-ad6bb2822f83\") "
Sep 11 14:03:43.991776 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.991073    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:43.991776 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.991708    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-data" (OuterVolumeSpecName: "istio-data") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:43.992534 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.992315    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:43.992867 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.992808    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:43.993262 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.993212    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-token" (OuterVolumeSpecName: "istio-token") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:43.993399 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.993229    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:03:43.994033 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.993999    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-kube-api-access-l8gtq" (OuterVolumeSpecName: "kube-api-access-l8gtq") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "kube-api-access-l8gtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:43.994694 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.994661    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:43.996559 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:43.996527    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e1443bee-1c53-4e1b-b96d-ad6bb2822f83" (UID: "e1443bee-1c53-4e1b-b96d-ad6bb2822f83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.091918    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.091947    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.091957    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8gtq\" (UniqueName: \"kubernetes.io/projected/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-kube-api-access-l8gtq\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.091968    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.091976    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.091983    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.091990    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.091998    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.092010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.092006    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1443bee-1c53-4e1b-b96d-ad6bb2822f83-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:44.332431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.332396    2563 generic.go:356] "Generic (PLEG): container finished" podID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerID="97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397" exitCode=0
Sep 11 14:03:44.332923 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.332483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" event={"ID":"e1443bee-1c53-4e1b-b96d-ad6bb2822f83","Type":"ContainerDied","Data":"97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397"}
Sep 11 14:03:44.332923 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.332505    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"
Sep 11 14:03:44.332923 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.332554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs" event={"ID":"e1443bee-1c53-4e1b-b96d-ad6bb2822f83","Type":"ContainerDied","Data":"0352548151f95940014b56d4f77dce0f685215dbc8d4363619dcaa44aed4b30b"}
Sep 11 14:03:44.332923 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.332605    2563 scope.go:117] "RemoveContainer" containerID="97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397"
Sep 11 14:03:44.342096 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.342052    2563 scope.go:117] "RemoveContainer" containerID="ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d"
Sep 11 14:03:44.351028 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.351005    2563 scope.go:117] "RemoveContainer" containerID="97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397"
Sep 11 14:03:44.351362 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:03:44.351342    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397\": container with ID starting with 97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397 not found: ID does not exist" containerID="97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397"
Sep 11 14:03:44.351406 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.351373    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397"} err="failed to get container status \"97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397\": rpc error: code = NotFound desc = could not find container \"97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397\": container with ID starting with 97edf986adce8b69bec6342f8f0802ac0bb4af5a7927d7bbd67e0e995416a397 not found: ID does not exist"
Sep 11 14:03:44.351406 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.351392    2563 scope.go:117] "RemoveContainer" containerID="ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d"
Sep 11 14:03:44.351655 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:03:44.351638    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d\": container with ID starting with ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d not found: ID does not exist" containerID="ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d"
Sep 11 14:03:44.351716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.351657    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d"} err="failed to get container status \"ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d\": rpc error: code = NotFound desc = could not find container \"ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d\": container with ID starting with ac72fe5628cb3fa8a9d3cc5e5f64a3f67ab1f448f72927ef016963b1a661263d not found: ID does not exist"
Sep 11 14:03:44.366016 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.365981    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"]
Sep 11 14:03:44.379460 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:44.379430    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bd7f15a9-openshift-default-79749d5fb6-wzmgs"]
Sep 11 14:03:46.240591 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:46.240555    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" path="/var/lib/kubelet/pods/e1443bee-1c53-4e1b-b96d-ad6bb2822f83/volumes"
Sep 11 14:03:48.361599 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.361567    2563 generic.go:356] "Generic (PLEG): container finished" podID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerID="d1d484db47a48f4869a973ef0d9c744927c3a591a3ad07b193ee13480cb87e19" exitCode=0
Sep 11 14:03:48.362096 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.361637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" event={"ID":"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa","Type":"ContainerDied","Data":"d1d484db47a48f4869a973ef0d9c744927c3a591a3ad07b193ee13480cb87e19"}
Sep 11 14:03:48.362096 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.361706    2563 scope.go:117] "RemoveContainer" containerID="a953ecee4d0048d97b4575844d881a4486d8a60297cb87f7cf951463016e7847"
Sep 11 14:03:48.437812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.437734    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:03:48.544328 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-envoy\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544328 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544329    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-data\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544591 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544358    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istiod-ca-cert\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544591 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544391    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv4dr\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-kube-api-access-sv4dr\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544591 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544415    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-token\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544591 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544548    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-socket\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544851 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-podinfo\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544851 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544654    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:48.544851 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544706    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-certs\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544851 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544742    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-credential-socket\") pod \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\" (UID: \"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa\") "
Sep 11 14:03:48.544851 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544829    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:48.545092 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544858    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:03:48.545092 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.544948    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:48.545192 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.545097    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:48.545243 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.545230    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.545299 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.545252    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.545299 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.545267    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.545299 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.545280    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.545299 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.545294    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.547040 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.547012    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:03:48.547040 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.547024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:48.547211 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.547183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:48.547248 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.547216    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-kube-api-access-sv4dr" (OuterVolumeSpecName: "kube-api-access-sv4dr") pod "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" (UID: "d1c7cd44-40f5-4444-a7e2-9fd109c0faaa"). InnerVolumeSpecName "kube-api-access-sv4dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:48.646468 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.646432    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.646468 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.646462    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv4dr\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-kube-api-access-sv4dr\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.646468 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.646472    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.646468 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.646480    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:03:48.748624 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:48.748531    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:03:49.370196 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:49.370165    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"
Sep 11 14:03:49.370701 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:49.370213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw" event={"ID":"d1c7cd44-40f5-4444-a7e2-9fd109c0faaa","Type":"ContainerDied","Data":"5f2530dffc3b5365e0c68d8e172e1e2b77adc4602617ff35815c7b5cfea8b842"}
Sep 11 14:03:49.370701 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:49.370270    2563 scope.go:117] "RemoveContainer" containerID="d1d484db47a48f4869a973ef0d9c744927c3a591a3ad07b193ee13480cb87e19"
Sep 11 14:03:49.394335 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:49.394308    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"]
Sep 11 14:03:49.400653 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:49.400631    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-498d98-openshift-default-578d9d7866-gmktw"]
Sep 11 14:03:50.244490 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:03:50.244452    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" path="/var/lib/kubelet/pods/d1c7cd44-40f5-4444-a7e2-9fd109c0faaa/volumes"
Sep 11 14:04:00.245931 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:00.245900    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"]
Sep 11 14:04:06.317142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317105    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"]
Sep 11 14:04:06.317537 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317495    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy"
Sep 11 14:04:06.317537 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317507    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy"
Sep 11 14:04:06.317537 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317517    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy"
Sep 11 14:04:06.317537 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317522    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy"
Sep 11 14:04:06.317537 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317540    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317546    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317552    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="391fc17e-7feb-4aa2-8dda-02abd78128a9" containerName="main"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317556    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="391fc17e-7feb-4aa2-8dda-02abd78128a9" containerName="main"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317569    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317574    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317580    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f3359d3-f4df-4587-95d5-53cd3049bb17" containerName="revoke-keys"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317584    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f3359d3-f4df-4587-95d5-53cd3049bb17" containerName="revoke-keys"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317643    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317650    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1c7cd44-40f5-4444-a7e2-9fd109c0faaa" containerName="istio-proxy"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317657    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="391fc17e-7feb-4aa2-8dda-02abd78128a9" containerName="main"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317667    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f3359d3-f4df-4587-95d5-53cd3049bb17" containerName="revoke-keys"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317674    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy"
Sep 11 14:04:06.317699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.317679    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1443bee-1c53-4e1b-b96d-ad6bb2822f83" containerName="istio-proxy"
Sep 11 14:04:06.321100 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.321082    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"
Sep 11 14:04:06.323569 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.323544    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:04:06.323715 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.323670    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:04:06.329252 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.328814    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"]
Sep 11 14:04:06.397626 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.397584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ada9afc-e1e0-45d5-87f0-3a82458f1313-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-10d06905-bpsql\" (UID: \"3ada9afc-e1e0-45d5-87f0-3a82458f1313\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"
Sep 11 14:04:06.498240 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.498199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ada9afc-e1e0-45d5-87f0-3a82458f1313-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-10d06905-bpsql\" (UID: \"3ada9afc-e1e0-45d5-87f0-3a82458f1313\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"
Sep 11 14:04:06.498871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.498852    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ada9afc-e1e0-45d5-87f0-3a82458f1313-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-10d06905-bpsql\" (UID: \"3ada9afc-e1e0-45d5-87f0-3a82458f1313\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"
Sep 11 14:04:06.633871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.633829    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"
Sep 11 14:04:06.905345 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:06.905318    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"]
Sep 11 14:04:06.907783 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:04:06.907737    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ada9afc_e1e0_45d5_87f0_3a82458f1313.slice/crio-780e7614e05e971738debef2b04d95457a2b9109312e4bd9faed4e88b56d4f23 WatchSource:0}: Error finding container 780e7614e05e971738debef2b04d95457a2b9109312e4bd9faed4e88b56d4f23: Status 404 returned error can't find the container with id 780e7614e05e971738debef2b04d95457a2b9109312e4bd9faed4e88b56d4f23
Sep 11 14:04:07.485877 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:07.485781    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql" event={"ID":"3ada9afc-e1e0-45d5-87f0-3a82458f1313","Type":"ContainerStarted","Data":"acd4b051187b944a60535b0a5a1ee952b6b0f21cf3a0c271eeb49061bdf42ea0"}
Sep 11 14:04:07.485877 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:07.485837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql" event={"ID":"3ada9afc-e1e0-45d5-87f0-3a82458f1313","Type":"ContainerStarted","Data":"780e7614e05e971738debef2b04d95457a2b9109312e4bd9faed4e88b56d4f23"}
Sep 11 14:04:07.504396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:07.504336    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql" podStartSLOduration=1.5043202180000002 podStartE2EDuration="1.504320218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:04:07.503171347 +0000 UTC m=+943.761371358" watchObservedRunningTime="2026-09-11 14:04:07.504320218 +0000 UTC m=+943.762520227"
Sep 11 14:04:08.493120 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:08.493083    2563 generic.go:356] "Generic (PLEG): container finished" podID="3ada9afc-e1e0-45d5-87f0-3a82458f1313" containerID="acd4b051187b944a60535b0a5a1ee952b6b0f21cf3a0c271eeb49061bdf42ea0" exitCode=0
Sep 11 14:04:08.493510 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:08.493170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql" event={"ID":"3ada9afc-e1e0-45d5-87f0-3a82458f1313","Type":"ContainerDied","Data":"acd4b051187b944a60535b0a5a1ee952b6b0f21cf3a0c271eeb49061bdf42ea0"}
Sep 11 14:04:09.641547 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:09.641516    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"
Sep 11 14:04:09.730478 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:09.730441    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ada9afc-e1e0-45d5-87f0-3a82458f1313-maas-api-service-ca\") pod \"3ada9afc-e1e0-45d5-87f0-3a82458f1313\" (UID: \"3ada9afc-e1e0-45d5-87f0-3a82458f1313\") "
Sep 11 14:04:09.730888 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:09.730864    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ada9afc-e1e0-45d5-87f0-3a82458f1313-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3ada9afc-e1e0-45d5-87f0-3a82458f1313" (UID: "3ada9afc-e1e0-45d5-87f0-3a82458f1313"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:04:09.832146 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:09.832052    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ada9afc-e1e0-45d5-87f0-3a82458f1313-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:04:10.362788 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.362737    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"]
Sep 11 14:04:10.363159 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.363148    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ada9afc-e1e0-45d5-87f0-3a82458f1313" containerName="revoke-keys"
Sep 11 14:04:10.363203 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.363161    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ada9afc-e1e0-45d5-87f0-3a82458f1313" containerName="revoke-keys"
Sep 11 14:04:10.363239 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.363225    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ada9afc-e1e0-45d5-87f0-3a82458f1313" containerName="revoke-keys"
Sep 11 14:04:10.365374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.365359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.369304 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.369274    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-da9534\"/\"test-model-da9534-kserve-self-signed-certs\""
Sep 11 14:04:10.369477 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.369276    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-da9534\"/\"default-dockercfg-c42pk\""
Sep 11 14:04:10.369477 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.369276    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-da9534\"/\"openshift-service-ca.crt\""
Sep 11 14:04:10.369477 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.369336    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-da9534\"/\"kube-root-ca.crt\""
Sep 11 14:04:10.382890 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.380276    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"]
Sep 11 14:04:10.438387 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.438350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tmp-dir\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.438563 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.438394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-home\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.438563 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.438494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-model-cache\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.438563 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.438535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-dshm\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.438743 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.438629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tls-certs\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.438743 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.438672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5z6b\" (UniqueName: \"kubernetes.io/projected/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-kube-api-access-p5z6b\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.508649 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.508575    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"
Sep 11 14:04:10.508649 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.508610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql" event={"ID":"3ada9afc-e1e0-45d5-87f0-3a82458f1313","Type":"ContainerDied","Data":"780e7614e05e971738debef2b04d95457a2b9109312e4bd9faed4e88b56d4f23"}
Sep 11 14:04:10.508867 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.508693    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="780e7614e05e971738debef2b04d95457a2b9109312e4bd9faed4e88b56d4f23"
Sep 11 14:04:10.539187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tmp-dir\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.539187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-home\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.539457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-model-cache\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.539457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-dshm\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.539457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tls-certs\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.539457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5z6b\" (UniqueName: \"kubernetes.io/projected/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-kube-api-access-p5z6b\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.539680 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tmp-dir\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.539680 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-model-cache\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.539805 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.539737    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-home\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.541861 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.541835    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-dshm\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.542094 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.542075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tls-certs\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.548073 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.548039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5z6b\" (UniqueName: \"kubernetes.io/projected/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-kube-api-access-p5z6b\") pod \"test-model-da9534-kserve-879bdb945-8ksv9\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") " pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.683827 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.683778    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:10.961376 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:10.961350    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"]
Sep 11 14:04:10.963249 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:04:10.963211    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4c5cab8_d00f_4dd2_8511_892a5e9197a1.slice/crio-ff8ecdbff40aaa485739875569fc5bb486f6a6032d4aab98cb3c923580fe81b8 WatchSource:0}: Error finding container ff8ecdbff40aaa485739875569fc5bb486f6a6032d4aab98cb3c923580fe81b8: Status 404 returned error can't find the container with id ff8ecdbff40aaa485739875569fc5bb486f6a6032d4aab98cb3c923580fe81b8
Sep 11 14:04:11.516952 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:11.516918    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9" event={"ID":"f4c5cab8-d00f-4dd2-8511-892a5e9197a1","Type":"ContainerStarted","Data":"4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05"}
Sep 11 14:04:11.517145 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:11.516960    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9" event={"ID":"f4c5cab8-d00f-4dd2-8511-892a5e9197a1","Type":"ContainerStarted","Data":"ff8ecdbff40aaa485739875569fc5bb486f6a6032d4aab98cb3c923580fe81b8"}
Sep 11 14:04:11.535371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:11.535322    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9" podStartSLOduration=1.535305867 podStartE2EDuration="1.535305867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:04:11.532375159 +0000 UTC m=+947.790575168" watchObservedRunningTime="2026-09-11 14:04:11.535305867 +0000 UTC m=+947.793505876"
Sep 11 14:04:12.151912 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:12.151876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:04:13.927043 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:13.927010    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"]
Sep 11 14:04:13.929453 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:13.929430    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10d06905-bpsql"]
Sep 11 14:04:14.241296 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:14.241216    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ada9afc-e1e0-45d5-87f0-3a82458f1313" path="/var/lib/kubelet/pods/3ada9afc-e1e0-45d5-87f0-3a82458f1313/volumes"
Sep 11 14:04:15.266821 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.266775    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6" podUID="06caca19-cde9-4f27-baef-fddea3627cb5" containerName="main" containerID="cri-o://7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796" gracePeriod=45
Sep 11 14:04:15.516811 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.516772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:04:15.547218 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.547128    2563 generic.go:356] "Generic (PLEG): container finished" podID="06caca19-cde9-4f27-baef-fddea3627cb5" containerID="7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796" exitCode=0
Sep 11 14:04:15.547218 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.547164    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"
Sep 11 14:04:15.547218 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.547203    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6" event={"ID":"06caca19-cde9-4f27-baef-fddea3627cb5","Type":"ContainerDied","Data":"7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796"}
Sep 11 14:04:15.547524 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.547249    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6" event={"ID":"06caca19-cde9-4f27-baef-fddea3627cb5","Type":"ContainerDied","Data":"4d2c63ecb61fc0cda57a02f12edae16ef103e2f2188473fee95878e1b59b9115"}
Sep 11 14:04:15.547524 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.547270    2563 scope.go:117] "RemoveContainer" containerID="7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796"
Sep 11 14:04:15.556942 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.556918    2563 scope.go:117] "RemoveContainer" containerID="7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796"
Sep 11 14:04:15.557340 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:04:15.557319    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796\": container with ID starting with 7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796 not found: ID does not exist" containerID="7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796"
Sep 11 14:04:15.557462 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.557350    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796"} err="failed to get container status \"7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796\": rpc error: code = NotFound desc = could not find container \"7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796\": container with ID starting with 7b3fe19f2d8b384d6f4a100895d5460ff8514b514111231e1339eeeb316d2796 not found: ID does not exist"
Sep 11 14:04:15.593944 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.593908    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h8rl\" (UniqueName: \"kubernetes.io/projected/06caca19-cde9-4f27-baef-fddea3627cb5-kube-api-access-5h8rl\") pod \"06caca19-cde9-4f27-baef-fddea3627cb5\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") "
Sep 11 14:04:15.594142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.593963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-tmp-dir\") pod \"06caca19-cde9-4f27-baef-fddea3627cb5\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") "
Sep 11 14:04:15.594142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.593999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-dshm\") pod \"06caca19-cde9-4f27-baef-fddea3627cb5\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") "
Sep 11 14:04:15.594142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.594042    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06caca19-cde9-4f27-baef-fddea3627cb5-tls-certs\") pod \"06caca19-cde9-4f27-baef-fddea3627cb5\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") "
Sep 11 14:04:15.594142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.594067    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-model-cache\") pod \"06caca19-cde9-4f27-baef-fddea3627cb5\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") "
Sep 11 14:04:15.594142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.594090    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-home\") pod \"06caca19-cde9-4f27-baef-fddea3627cb5\" (UID: \"06caca19-cde9-4f27-baef-fddea3627cb5\") "
Sep 11 14:04:15.594412 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.594308    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06caca19-cde9-4f27-baef-fddea3627cb5" (UID: "06caca19-cde9-4f27-baef-fddea3627cb5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:15.594412 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.594327    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-model-cache" (OuterVolumeSpecName: "model-cache") pod "06caca19-cde9-4f27-baef-fddea3627cb5" (UID: "06caca19-cde9-4f27-baef-fddea3627cb5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:15.594500 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.594421    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:04:15.594500 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.594425    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-home" (OuterVolumeSpecName: "home") pod "06caca19-cde9-4f27-baef-fddea3627cb5" (UID: "06caca19-cde9-4f27-baef-fddea3627cb5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:15.596424 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.596395    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-dshm" (OuterVolumeSpecName: "dshm") pod "06caca19-cde9-4f27-baef-fddea3627cb5" (UID: "06caca19-cde9-4f27-baef-fddea3627cb5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:15.596547 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.596418    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06caca19-cde9-4f27-baef-fddea3627cb5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06caca19-cde9-4f27-baef-fddea3627cb5" (UID: "06caca19-cde9-4f27-baef-fddea3627cb5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:04:15.596547 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.596447    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06caca19-cde9-4f27-baef-fddea3627cb5-kube-api-access-5h8rl" (OuterVolumeSpecName: "kube-api-access-5h8rl") pod "06caca19-cde9-4f27-baef-fddea3627cb5" (UID: "06caca19-cde9-4f27-baef-fddea3627cb5"). InnerVolumeSpecName "kube-api-access-5h8rl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:04:15.695237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.695196    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:04:15.695237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.695229    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06caca19-cde9-4f27-baef-fddea3627cb5-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:04:15.695237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.695242    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:04:15.695237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.695250    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06caca19-cde9-4f27-baef-fddea3627cb5-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:04:15.695522 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.695258    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h8rl\" (UniqueName: \"kubernetes.io/projected/06caca19-cde9-4f27-baef-fddea3627cb5-kube-api-access-5h8rl\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:04:15.869453 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.869419    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"]
Sep 11 14:04:15.873360 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:15.873330    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-56cecb57-kserve-6b58b74675-8v6d6"]
Sep 11 14:04:16.241405 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:16.241324    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06caca19-cde9-4f27-baef-fddea3627cb5" path="/var/lib/kubelet/pods/06caca19-cde9-4f27-baef-fddea3627cb5/volumes"
Sep 11 14:04:20.684194 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:20.684158    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:20.684635 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:20.684403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:20.697239 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:20.697211    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:21.594780 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:21.594737    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:04:31.133781 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:31.133708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:04:47.437000 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:04:47.436904    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:05:03.937994 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:03.937955    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:05:07.670138 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.670085    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"]
Sep 11 14:05:07.670549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.670514    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06caca19-cde9-4f27-baef-fddea3627cb5" containerName="main"
Sep 11 14:05:07.670549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.670528    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06caca19-cde9-4f27-baef-fddea3627cb5" containerName="main"
Sep 11 14:05:07.670623 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.670615    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="06caca19-cde9-4f27-baef-fddea3627cb5" containerName="main"
Sep 11 14:05:07.673901 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.673884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"
Sep 11 14:05:07.676612 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.676587    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:05:07.676803 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.676646    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:05:07.685261 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.685228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"]
Sep 11 14:05:07.825373 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.825336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a9d13-f7a9-45d7-926e-a1f383f3d46e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-93da06-p28p9\" (UID: \"c45a9d13-f7a9-45d7-926e-a1f383f3d46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"
Sep 11 14:05:07.926357 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.926257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a9d13-f7a9-45d7-926e-a1f383f3d46e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-93da06-p28p9\" (UID: \"c45a9d13-f7a9-45d7-926e-a1f383f3d46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"
Sep 11 14:05:07.926969 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.926948    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a9d13-f7a9-45d7-926e-a1f383f3d46e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-93da06-p28p9\" (UID: \"c45a9d13-f7a9-45d7-926e-a1f383f3d46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"
Sep 11 14:05:07.986308 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:07.986263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"
Sep 11 14:05:08.264263 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:08.264236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"]
Sep 11 14:05:08.265929 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:05:08.265892    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45a9d13_f7a9_45d7_926e_a1f383f3d46e.slice/crio-6a591f5c1d2239ed04c842ed1af6cec2836ca2bdbe5b472cb7b28a5f5cb9ab5b WatchSource:0}: Error finding container 6a591f5c1d2239ed04c842ed1af6cec2836ca2bdbe5b472cb7b28a5f5cb9ab5b: Status 404 returned error can't find the container with id 6a591f5c1d2239ed04c842ed1af6cec2836ca2bdbe5b472cb7b28a5f5cb9ab5b
Sep 11 14:05:08.879745 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:08.879701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9" event={"ID":"c45a9d13-f7a9-45d7-926e-a1f383f3d46e","Type":"ContainerStarted","Data":"8e1985cf0f2f9be100b352ce0c916bccf60fb6e7c5e6afa09e908d178cb638e1"}
Sep 11 14:05:08.879745 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:08.879779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9" event={"ID":"c45a9d13-f7a9-45d7-926e-a1f383f3d46e","Type":"ContainerStarted","Data":"6a591f5c1d2239ed04c842ed1af6cec2836ca2bdbe5b472cb7b28a5f5cb9ab5b"}
Sep 11 14:05:08.895185 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:08.895123    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9" podStartSLOduration=1.895107391 podStartE2EDuration="1.895107391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:08.893734814 +0000 UTC m=+1005.151934864" watchObservedRunningTime="2026-09-11 14:05:08.895107391 +0000 UTC m=+1005.153307399"
Sep 11 14:05:09.887537 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:09.887501    2563 generic.go:356] "Generic (PLEG): container finished" podID="c45a9d13-f7a9-45d7-926e-a1f383f3d46e" containerID="8e1985cf0f2f9be100b352ce0c916bccf60fb6e7c5e6afa09e908d178cb638e1" exitCode=0
Sep 11 14:05:09.887978 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:09.887583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9" event={"ID":"c45a9d13-f7a9-45d7-926e-a1f383f3d46e","Type":"ContainerDied","Data":"8e1985cf0f2f9be100b352ce0c916bccf60fb6e7c5e6afa09e908d178cb638e1"}
Sep 11 14:05:11.036779 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:11.036726    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"
Sep 11 14:05:11.162378 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:11.162265    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a9d13-f7a9-45d7-926e-a1f383f3d46e-maas-api-service-ca\") pod \"c45a9d13-f7a9-45d7-926e-a1f383f3d46e\" (UID: \"c45a9d13-f7a9-45d7-926e-a1f383f3d46e\") "
Sep 11 14:05:11.162706 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:11.162671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c45a9d13-f7a9-45d7-926e-a1f383f3d46e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c45a9d13-f7a9-45d7-926e-a1f383f3d46e" (UID: "c45a9d13-f7a9-45d7-926e-a1f383f3d46e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:05:11.263523 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:11.263465    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a9d13-f7a9-45d7-926e-a1f383f3d46e-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:11.908108 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:11.908074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9" event={"ID":"c45a9d13-f7a9-45d7-926e-a1f383f3d46e","Type":"ContainerDied","Data":"6a591f5c1d2239ed04c842ed1af6cec2836ca2bdbe5b472cb7b28a5f5cb9ab5b"}
Sep 11 14:05:11.908108 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:11.908102    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"
Sep 11 14:05:11.908346 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:11.908155    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a591f5c1d2239ed04c842ed1af6cec2836ca2bdbe5b472cb7b28a5f5cb9ab5b"
Sep 11 14:05:18.397071 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:18.397022    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"]
Sep 11 14:05:18.399444 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:18.399420    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-93da06-p28p9"]
Sep 11 14:05:18.838991 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:18.838901    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:05:19.769958 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:19.769918    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"]
Sep 11 14:05:20.240456 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:20.240420    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c45a9d13-f7a9-45d7-926e-a1f383f3d46e" path="/var/lib/kubelet/pods/c45a9d13-f7a9-45d7-926e-a1f383f3d46e/volumes"
Sep 11 14:05:20.699497 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:20.699459    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"]
Sep 11 14:05:29.833108 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:29.833074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:05:34.790743 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:34.790676    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt" podUID="d544fc62-df68-4c44-8357-b17f69958b75" containerName="main" containerID="cri-o://b699e3718cf52f9e8dfa6e4bd591ecc92b85df9da22c274043edb17dfe6a68db" gracePeriod=45
Sep 11 14:05:35.049292 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.049259    2563 generic.go:356] "Generic (PLEG): container finished" podID="d544fc62-df68-4c44-8357-b17f69958b75" containerID="b699e3718cf52f9e8dfa6e4bd591ecc92b85df9da22c274043edb17dfe6a68db" exitCode=0
Sep 11 14:05:35.049480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.049339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt" event={"ID":"d544fc62-df68-4c44-8357-b17f69958b75","Type":"ContainerDied","Data":"b699e3718cf52f9e8dfa6e4bd591ecc92b85df9da22c274043edb17dfe6a68db"}
Sep 11 14:05:35.049480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.049384    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt" event={"ID":"d544fc62-df68-4c44-8357-b17f69958b75","Type":"ContainerDied","Data":"acaf797231bd823d6491d7c471fb1ee3585465f20f164d61b7986b6b282d8f81"}
Sep 11 14:05:35.049480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.049396    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acaf797231bd823d6491d7c471fb1ee3585465f20f164d61b7986b6b282d8f81"
Sep 11 14:05:35.059220 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.059192    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:05:35.201586 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.201551    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-home\") pod \"d544fc62-df68-4c44-8357-b17f69958b75\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") "
Sep 11 14:05:35.201847 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.201648    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-tmp-dir\") pod \"d544fc62-df68-4c44-8357-b17f69958b75\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") "
Sep 11 14:05:35.201847 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.201690    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gshdp\" (UniqueName: \"kubernetes.io/projected/d544fc62-df68-4c44-8357-b17f69958b75-kube-api-access-gshdp\") pod \"d544fc62-df68-4c44-8357-b17f69958b75\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") "
Sep 11 14:05:35.201847 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.201783    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d544fc62-df68-4c44-8357-b17f69958b75-tls-certs\") pod \"d544fc62-df68-4c44-8357-b17f69958b75\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") "
Sep 11 14:05:35.201847 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.201840    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-model-cache\") pod \"d544fc62-df68-4c44-8357-b17f69958b75\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") "
Sep 11 14:05:35.202083 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.201882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-dshm\") pod \"d544fc62-df68-4c44-8357-b17f69958b75\" (UID: \"d544fc62-df68-4c44-8357-b17f69958b75\") "
Sep 11 14:05:35.202083 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.201901    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-home" (OuterVolumeSpecName: "home") pod "d544fc62-df68-4c44-8357-b17f69958b75" (UID: "d544fc62-df68-4c44-8357-b17f69958b75"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:35.202083 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.201980    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d544fc62-df68-4c44-8357-b17f69958b75" (UID: "d544fc62-df68-4c44-8357-b17f69958b75"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:35.202225 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.202160    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-model-cache" (OuterVolumeSpecName: "model-cache") pod "d544fc62-df68-4c44-8357-b17f69958b75" (UID: "d544fc62-df68-4c44-8357-b17f69958b75"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:35.202298 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.202281    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:35.202354 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.202303    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:35.202354 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.202316    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:35.204395 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.204368    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-dshm" (OuterVolumeSpecName: "dshm") pod "d544fc62-df68-4c44-8357-b17f69958b75" (UID: "d544fc62-df68-4c44-8357-b17f69958b75"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:35.204395 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.204385    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d544fc62-df68-4c44-8357-b17f69958b75-kube-api-access-gshdp" (OuterVolumeSpecName: "kube-api-access-gshdp") pod "d544fc62-df68-4c44-8357-b17f69958b75" (UID: "d544fc62-df68-4c44-8357-b17f69958b75"). InnerVolumeSpecName "kube-api-access-gshdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:35.204548 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.204412    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d544fc62-df68-4c44-8357-b17f69958b75-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d544fc62-df68-4c44-8357-b17f69958b75" (UID: "d544fc62-df68-4c44-8357-b17f69958b75"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:35.303946 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.303849    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gshdp\" (UniqueName: \"kubernetes.io/projected/d544fc62-df68-4c44-8357-b17f69958b75-kube-api-access-gshdp\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:35.303946 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.303885    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d544fc62-df68-4c44-8357-b17f69958b75-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:35.303946 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.303899    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d544fc62-df68-4c44-8357-b17f69958b75-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:35.721023 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:35.720982    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9" podUID="f4c5cab8-d00f-4dd2-8511-892a5e9197a1" containerName="main" containerID="cri-o://4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05" gracePeriod=45
Sep 11 14:05:36.010304 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.010277    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:05:36.056899 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.056861    2563 generic.go:356] "Generic (PLEG): container finished" podID="f4c5cab8-d00f-4dd2-8511-892a5e9197a1" containerID="4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05" exitCode=0
Sep 11 14:05:36.057089 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.056926    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"
Sep 11 14:05:36.057089 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.056953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9" event={"ID":"f4c5cab8-d00f-4dd2-8511-892a5e9197a1","Type":"ContainerDied","Data":"4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05"}
Sep 11 14:05:36.057089 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.057002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9" event={"ID":"f4c5cab8-d00f-4dd2-8511-892a5e9197a1","Type":"ContainerDied","Data":"ff8ecdbff40aaa485739875569fc5bb486f6a6032d4aab98cb3c923580fe81b8"}
Sep 11 14:05:36.057089 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.057022    2563 scope.go:117] "RemoveContainer" containerID="4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05"
Sep 11 14:05:36.057286 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.057190    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"
Sep 11 14:05:36.067811 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.067784    2563 scope.go:117] "RemoveContainer" containerID="4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05"
Sep 11 14:05:36.068150 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:05:36.068128    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05\": container with ID starting with 4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05 not found: ID does not exist" containerID="4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05"
Sep 11 14:05:36.068210 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.068158    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05"} err="failed to get container status \"4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05\": rpc error: code = NotFound desc = could not find container \"4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05\": container with ID starting with 4350fd68b92a1ac76c8c6f660cace5d20f8ad8be4ebb0d6aa18c5f77d2298e05 not found: ID does not exist"
Sep 11 14:05:36.079966 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.079937    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"]
Sep 11 14:05:36.084740 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.084714    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9c9fc1/test-model-9c9fc1-kserve-6dfc56f84c-fftgt"]
Sep 11 14:05:36.112687 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.112654    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tmp-dir\") pod \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") "
Sep 11 14:05:36.112687 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.112688    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-model-cache\") pod \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") "
Sep 11 14:05:36.112936 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.112712    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-home\") pod \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") "
Sep 11 14:05:36.112936 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.112736    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5z6b\" (UniqueName: \"kubernetes.io/projected/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-kube-api-access-p5z6b\") pod \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") "
Sep 11 14:05:36.112936 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.112809    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-dshm\") pod \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") "
Sep 11 14:05:36.112936 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.112839    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tls-certs\") pod \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\" (UID: \"f4c5cab8-d00f-4dd2-8511-892a5e9197a1\") "
Sep 11 14:05:36.113140 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.113034    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-home" (OuterVolumeSpecName: "home") pod "f4c5cab8-d00f-4dd2-8511-892a5e9197a1" (UID: "f4c5cab8-d00f-4dd2-8511-892a5e9197a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:36.113140 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.113042    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "f4c5cab8-d00f-4dd2-8511-892a5e9197a1" (UID: "f4c5cab8-d00f-4dd2-8511-892a5e9197a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:36.113140 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.113064    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f4c5cab8-d00f-4dd2-8511-892a5e9197a1" (UID: "f4c5cab8-d00f-4dd2-8511-892a5e9197a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:36.113276 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.113160    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:36.113276 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.113183    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:36.113276 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.113197    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:36.115386 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.115354    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f4c5cab8-d00f-4dd2-8511-892a5e9197a1" (UID: "f4c5cab8-d00f-4dd2-8511-892a5e9197a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:36.115386 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.115358    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-dshm" (OuterVolumeSpecName: "dshm") pod "f4c5cab8-d00f-4dd2-8511-892a5e9197a1" (UID: "f4c5cab8-d00f-4dd2-8511-892a5e9197a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:36.115386 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.115365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-kube-api-access-p5z6b" (OuterVolumeSpecName: "kube-api-access-p5z6b") pod "f4c5cab8-d00f-4dd2-8511-892a5e9197a1" (UID: "f4c5cab8-d00f-4dd2-8511-892a5e9197a1"). InnerVolumeSpecName "kube-api-access-p5z6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:36.214171 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.214130    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5z6b\" (UniqueName: \"kubernetes.io/projected/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-kube-api-access-p5z6b\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:36.214171 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.214166    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:36.214171 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.214179    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4c5cab8-d00f-4dd2-8511-892a5e9197a1-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:36.240525 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.240434    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d544fc62-df68-4c44-8357-b17f69958b75" path="/var/lib/kubelet/pods/d544fc62-df68-4c44-8357-b17f69958b75/volumes"
Sep 11 14:05:36.376212 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.376173    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"]
Sep 11 14:05:36.380503 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:36.380467    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-da9534/test-model-da9534-kserve-879bdb945-8ksv9"]
Sep 11 14:05:38.240702 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:38.240661    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4c5cab8-d00f-4dd2-8511-892a5e9197a1" path="/var/lib/kubelet/pods/f4c5cab8-d00f-4dd2-8511-892a5e9197a1/volumes"
Sep 11 14:05:38.738985 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:38.738946    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"]
Sep 11 14:05:38.739255 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:38.739228    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy" containerID="cri-o://24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad" gracePeriod=30
Sep 11 14:05:39.940264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:39.940227    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:05:43.315836 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:43.315798    2563 patch_prober.go:28] interesting pod/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:05:43.316202 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:43.315855    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:05:43.996302 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:43.996270    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:05:44.090071 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.089970    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-credential-socket\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090071 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090019    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-data\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090071 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-socket\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090107    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-certs\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090151    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-podinfo\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090177    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3163d52a-3ce4-4250-88b0-e106e65c816b-istiod-ca-cert\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090202    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9594p\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-kube-api-access-9594p\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090226    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-envoy\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090269    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-token\") pod \"3163d52a-3ce4-4250-88b0-e106e65c816b\" (UID: \"3163d52a-3ce4-4250-88b0-e106e65c816b\") "
Sep 11 14:05:44.090374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090337    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.090731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090378    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.090731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090568    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.090731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090610    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.090731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090565    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.090731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090583    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.090731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.090593    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3163d52a-3ce4-4250-88b0-e106e65c816b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:05:44.092722 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.092687    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.092894 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.092796    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:05:44.093008 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.092991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:44.093072 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.093039    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-kube-api-access-9594p" (OuterVolumeSpecName: "kube-api-access-9594p") pod "3163d52a-3ce4-4250-88b0-e106e65c816b" (UID: "3163d52a-3ce4-4250-88b0-e106e65c816b"). InnerVolumeSpecName "kube-api-access-9594p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:44.115571 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.115535    2563 generic.go:356] "Generic (PLEG): container finished" podID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerID="24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad" exitCode=0
Sep 11 14:05:44.115781 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.115592    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"
Sep 11 14:05:44.115781 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.115613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" event={"ID":"3163d52a-3ce4-4250-88b0-e106e65c816b","Type":"ContainerDied","Data":"24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad"}
Sep 11 14:05:44.115781 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.115702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp" event={"ID":"3163d52a-3ce4-4250-88b0-e106e65c816b","Type":"ContainerDied","Data":"9b6bf4675bf891b40fb67aeecfcc0ed6b10389379f9c348a34478b1c293300b6"}
Sep 11 14:05:44.115781 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.115734    2563 scope.go:117] "RemoveContainer" containerID="24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad"
Sep 11 14:05:44.126022 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.125994    2563 scope.go:117] "RemoveContainer" containerID="64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19"
Sep 11 14:05:44.135423 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.135392    2563 scope.go:117] "RemoveContainer" containerID="24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad"
Sep 11 14:05:44.135717 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:05:44.135700    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad\": container with ID starting with 24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad not found: ID does not exist" containerID="24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad"
Sep 11 14:05:44.135818 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.135727    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad"} err="failed to get container status \"24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad\": rpc error: code = NotFound desc = could not find container \"24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad\": container with ID starting with 24837916f810538e541b6fc96205f7c8c2545b2d11cea5fce52c2fab6d0fe6ad not found: ID does not exist"
Sep 11 14:05:44.135876 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.135744    2563 scope.go:117] "RemoveContainer" containerID="64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19"
Sep 11 14:05:44.136201 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:05:44.136178    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19\": container with ID starting with 64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19 not found: ID does not exist" containerID="64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19"
Sep 11 14:05:44.136363 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.136209    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19"} err="failed to get container status \"64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19\": rpc error: code = NotFound desc = could not find container \"64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19\": container with ID starting with 64dea39511b2d2f7b0c1fe078bbe5b6213d50f32590c2f718c95b1e5e6d38d19 not found: ID does not exist"
Sep 11 14:05:44.138961 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.138938    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"]
Sep 11 14:05:44.142383 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.142364    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-9c9fc1-openshift-default-747974446d-r4nwp"]
Sep 11 14:05:44.191465 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.191433    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.191465 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.191461    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.191465 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.191471    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3163d52a-3ce4-4250-88b0-e106e65c816b-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.191716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.191482    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9594p\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-kube-api-access-9594p\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.191716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.191491    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.191716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.191499    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3163d52a-3ce4-4250-88b0-e106e65c816b-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.191716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.191506    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3163d52a-3ce4-4250-88b0-e106e65c816b-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.241132 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:44.241103    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" path="/var/lib/kubelet/pods/3163d52a-3ce4-4250-88b0-e106e65c816b/volumes"
Sep 11 14:05:48.828651 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:48.828614    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:05:56.681870 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:56.681831    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"]
Sep 11 14:05:56.682309 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:56.682064    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy" containerID="cri-o://c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1" gracePeriod=30
Sep 11 14:05:57.131603 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.131562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:05:57.561875 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.561790    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"]
Sep 11 14:05:57.562174 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562162    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d544fc62-df68-4c44-8357-b17f69958b75" containerName="main"
Sep 11 14:05:57.562229 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562176    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d544fc62-df68-4c44-8357-b17f69958b75" containerName="main"
Sep 11 14:05:57.562229 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562188    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy"
Sep 11 14:05:57.562229 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562194    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy"
Sep 11 14:05:57.562229 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562207    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4c5cab8-d00f-4dd2-8511-892a5e9197a1" containerName="main"
Sep 11 14:05:57.562229 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562212    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4c5cab8-d00f-4dd2-8511-892a5e9197a1" containerName="main"
Sep 11 14:05:57.562229 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562223    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c45a9d13-f7a9-45d7-926e-a1f383f3d46e" containerName="revoke-keys"
Sep 11 14:05:57.562229 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562228    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c45a9d13-f7a9-45d7-926e-a1f383f3d46e" containerName="revoke-keys"
Sep 11 14:05:57.562467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562238    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy"
Sep 11 14:05:57.562467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562242    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy"
Sep 11 14:05:57.562467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562295    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4c5cab8-d00f-4dd2-8511-892a5e9197a1" containerName="main"
Sep 11 14:05:57.562467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562303    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy"
Sep 11 14:05:57.562467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562308    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d544fc62-df68-4c44-8357-b17f69958b75" containerName="main"
Sep 11 14:05:57.562467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.562315    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c45a9d13-f7a9-45d7-926e-a1f383f3d46e" containerName="revoke-keys"
Sep 11 14:05:57.564705 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.564684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.567402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.567374    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-932c97-openshift-default-dockercfg-fnh82\""
Sep 11 14:05:57.582329 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.582298    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"]
Sep 11 14:05:57.619665 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.619630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-data\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.619887 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.619672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-podinfo\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.619887 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.619735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-credential-socket\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.619887 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.619835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgqqz\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-kube-api-access-kgqqz\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.619887 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.619878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-socket\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.620061 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.619908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-token\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.620061 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.619965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istiod-ca-cert\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.620061 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.620014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-certs\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.620061 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.620047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-envoy\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720478 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-token\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istiod-ca-cert\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-certs\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-envoy\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-data\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-podinfo\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-credential-socket\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgqqz\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-kube-api-access-kgqqz\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.720972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.720707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-socket\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.721371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.721234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-certs\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.721371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.721275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-socket\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.721461 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.721431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-data\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.721521 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.721425    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-credential-socket\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.721929 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.721909    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istiod-ca-cert\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.723253 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.723230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-envoy\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.723489 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.723469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-podinfo\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.729053 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.729031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgqqz\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-kube-api-access-kgqqz\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.729330 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.729311    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-token\") pod \"e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") " pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:57.875222 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:57.875187    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:58.156132 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:58.156103    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"]
Sep 11 14:05:58.158639 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:05:58.158607    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd724c8eb_7e5b_4ce7_8860_b01f4d037837.slice/crio-fb6bdcd1acc79d56d5e02ec5213b809ef81555b9cfcec05d266ee65963390bca WatchSource:0}: Error finding container fb6bdcd1acc79d56d5e02ec5213b809ef81555b9cfcec05d266ee65963390bca: Status 404 returned error can't find the container with id fb6bdcd1acc79d56d5e02ec5213b809ef81555b9cfcec05d266ee65963390bca
Sep 11 14:05:58.161308 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:58.161280    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:58.161389 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:58.161347    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:58.161389 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:58.161378    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:58.199317 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:58.199283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" event={"ID":"d724c8eb-7e5b-4ce7-8860-b01f4d037837","Type":"ContainerStarted","Data":"fb6bdcd1acc79d56d5e02ec5213b809ef81555b9cfcec05d266ee65963390bca"}
Sep 11 14:05:59.207206 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:59.207169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" event={"ID":"d724c8eb-7e5b-4ce7-8860-b01f4d037837","Type":"ContainerStarted","Data":"3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2"}
Sep 11 14:05:59.228943 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:59.228877    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podStartSLOduration=2.228853278 podStartE2EDuration="2.228853278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:59.226784226 +0000 UTC m=+1055.484984237" watchObservedRunningTime="2026-09-11 14:05:59.228853278 +0000 UTC m=+1055.487053287"
Sep 11 14:05:59.875977 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:59.875938    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:05:59.881228 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:05:59.881202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:06:00.213763 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:00.213669    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:06:00.214675 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:00.214659    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:06:00.265094 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:00.265056    2563 patch_prober.go:28] interesting pod/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:06:00.265275 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:00.265119    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:06:01.943364 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.943334    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:06:01.958270 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958181    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-token\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958270 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958228    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-podinfo\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958294    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76mcg\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-kube-api-access-76mcg\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958336    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-credential-socket\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958356    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-data\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958373    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-envoy\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958409    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-certs\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958845 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958476    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-socket\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958845 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958503    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istiod-ca-cert\") pod \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\" (UID: \"99724b07-84c6-4aac-a38f-7ff0e89c29aa\") "
Sep 11 14:06:01.958845 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958676    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:01.958845 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958690    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:01.959078 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958954    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:01.959078 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958981    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:01.959078 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.958953    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:01.959078 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.959005    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:01.959078 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.959022    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:06:01.960713 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.960689    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:06:01.960918 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.960899    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:01.961107 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.961085    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:06:01.961181 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:01.961102    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-kube-api-access-76mcg" (OuterVolumeSpecName: "kube-api-access-76mcg") pod "99724b07-84c6-4aac-a38f-7ff0e89c29aa" (UID: "99724b07-84c6-4aac-a38f-7ff0e89c29aa"). InnerVolumeSpecName "kube-api-access-76mcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:06:02.060402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.060362    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76mcg\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-kube-api-access-76mcg\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:02.060402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.060398    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:02.060402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.060408    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:02.060652 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.060416    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99724b07-84c6-4aac-a38f-7ff0e89c29aa-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:02.060652 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.060426    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:02.060652 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.060434    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:02.060652 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.060441    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99724b07-84c6-4aac-a38f-7ff0e89c29aa-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:02.227102 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.227003    2563 generic.go:356] "Generic (PLEG): container finished" podID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerID="c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1" exitCode=0
Sep 11 14:06:02.227102 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.227064    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"
Sep 11 14:06:02.227102 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.227089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" event={"ID":"99724b07-84c6-4aac-a38f-7ff0e89c29aa","Type":"ContainerDied","Data":"c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1"}
Sep 11 14:06:02.227388 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.227173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m" event={"ID":"99724b07-84c6-4aac-a38f-7ff0e89c29aa","Type":"ContainerDied","Data":"3fb1b270980c591e4d57c59ffeefbdde74edc883e0a9d09f9e243130d57b7358"}
Sep 11 14:06:02.227388 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.227197    2563 scope.go:117] "RemoveContainer" containerID="c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1"
Sep 11 14:06:02.237889 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.237868    2563 scope.go:117] "RemoveContainer" containerID="a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd"
Sep 11 14:06:02.248280 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.248258    2563 scope.go:117] "RemoveContainer" containerID="c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1"
Sep 11 14:06:02.248628 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:06:02.248607    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1\": container with ID starting with c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1 not found: ID does not exist" containerID="c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1"
Sep 11 14:06:02.248719 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.248638    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1"} err="failed to get container status \"c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1\": rpc error: code = NotFound desc = could not find container \"c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1\": container with ID starting with c2c97ddd514c971381c5718be103177661186874d9b384c18c46abc0d8008dc1 not found: ID does not exist"
Sep 11 14:06:02.248719 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.248656    2563 scope.go:117] "RemoveContainer" containerID="a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd"
Sep 11 14:06:02.249013 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:06:02.248990    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd\": container with ID starting with a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd not found: ID does not exist" containerID="a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd"
Sep 11 14:06:02.249117 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.249017    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd"} err="failed to get container status \"a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd\": rpc error: code = NotFound desc = could not find container \"a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd\": container with ID starting with a25f455286f1048f99020386aec6a714f23eb65e06349787979da22ec3c21bfd not found: ID does not exist"
Sep 11 14:06:02.255664 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.255634    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"]
Sep 11 14:06:02.261034 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:02.261004    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-da9534-openshift-default-64db5bf964-fn54m"]
Sep 11 14:06:04.240621 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:04.240586    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" path="/var/lib/kubelet/pods/99724b07-84c6-4aac-a38f-7ff0e89c29aa/volumes"
Sep 11 14:06:07.938950 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:07.938907    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:06:16.255992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:16.255901    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 14:06:16.255992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:16.255973    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 14:06:17.323630 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:17.323530    2563 generic.go:356] "Generic (PLEG): container finished" podID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerID="3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2" exitCode=0
Sep 11 14:06:17.323630 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:17.323604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" event={"ID":"d724c8eb-7e5b-4ce7-8860-b01f4d037837","Type":"ContainerDied","Data":"3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2"}
Sep 11 14:06:17.324092 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:17.324014    2563 scope.go:117] "RemoveContainer" containerID="3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2"
Sep 11 14:06:17.325052 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:17.325024    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:17.325149 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:17.325089    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:17.325149 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:17.325120    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:18.332615 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:18.332571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" event={"ID":"d724c8eb-7e5b-4ce7-8860-b01f4d037837","Type":"ContainerStarted","Data":"8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849"}
Sep 11 14:06:18.876314 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:18.876271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:06:18.876561 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:18.876540    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 14:06:18.876611 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:18.876593    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 14:06:19.876373 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:19.876336    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 14:06:19.876832 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:19.876395    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 14:06:20.875636 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:20.875600    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 14:06:20.875854 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:20.875662    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 14:06:21.875848 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:21.875807    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 14:06:21.876263 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:21.875896    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 14:06:22.876401 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:22.876366    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 14:06:22.876836 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:22.876423    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 14:06:23.990816 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:23.990782    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:06:23.991246 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:23.991177    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:06:23.991962 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:23.991945    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:06:25.108162 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.108123    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"]
Sep 11 14:06:25.108578 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.108503    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy"
Sep 11 14:06:25.108578 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.108514    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy"
Sep 11 14:06:25.108578 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.108522    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy"
Sep 11 14:06:25.108578 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.108527    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy"
Sep 11 14:06:25.108717 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.108606    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy"
Sep 11 14:06:25.108717 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.108613    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="99724b07-84c6-4aac-a38f-7ff0e89c29aa" containerName="istio-proxy"
Sep 11 14:06:25.108717 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.108623    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3163d52a-3ce4-4250-88b0-e106e65c816b" containerName="istio-proxy"
Sep 11 14:06:25.113162 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.113140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"
Sep 11 14:06:25.116288 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.116255    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:06:25.116539 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.116523    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:06:25.129396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.129366    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"]
Sep 11 14:06:25.182145 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.182099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a78f950e-299a-43fd-a1e8-7627b0f298dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx\" (UID: \"a78f950e-299a-43fd-a1e8-7627b0f298dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"
Sep 11 14:06:25.283527 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.283482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a78f950e-299a-43fd-a1e8-7627b0f298dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx\" (UID: \"a78f950e-299a-43fd-a1e8-7627b0f298dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"
Sep 11 14:06:25.284176 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.284155    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a78f950e-299a-43fd-a1e8-7627b0f298dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx\" (UID: \"a78f950e-299a-43fd-a1e8-7627b0f298dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"
Sep 11 14:06:25.423679 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.423588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"
Sep 11 14:06:25.697041 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:25.697016    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"]
Sep 11 14:06:25.699669 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:06:25.699626    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda78f950e_299a_43fd_a1e8_7627b0f298dd.slice/crio-b89ee8997ce1ee2c1727bc9105b88b7a6ee7a383bb036b0db90cc815b2631446 WatchSource:0}: Error finding container b89ee8997ce1ee2c1727bc9105b88b7a6ee7a383bb036b0db90cc815b2631446: Status 404 returned error can't find the container with id b89ee8997ce1ee2c1727bc9105b88b7a6ee7a383bb036b0db90cc815b2631446
Sep 11 14:06:26.382462 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.382420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx" event={"ID":"a78f950e-299a-43fd-a1e8-7627b0f298dd","Type":"ContainerStarted","Data":"636c7521ec6dc6a356ba14eb441f21bee03f92fe4c85f45fd94a263b4a4f6b8a"}
Sep 11 14:06:26.382917 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.382478    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx" event={"ID":"a78f950e-299a-43fd-a1e8-7627b0f298dd","Type":"ContainerStarted","Data":"b89ee8997ce1ee2c1727bc9105b88b7a6ee7a383bb036b0db90cc815b2631446"}
Sep 11 14:06:26.399682 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.399631    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx" podStartSLOduration=1.399616266 podStartE2EDuration="1.399616266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:26.398100595 +0000 UTC m=+1082.656300622" watchObservedRunningTime="2026-09-11 14:06:26.399616266 +0000 UTC m=+1082.657816275"
Sep 11 14:06:26.702437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.702350    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"]
Sep 11 14:06:26.706460 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.706429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.709346 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.709321    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-367517-openshift-default-dockercfg-gv469\""
Sep 11 14:06:26.716829 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.716803    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"]
Sep 11 14:06:26.796931 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.796897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-token\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.797156 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.796951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc7a8bb-403d-4865-bf9d-94959070730b-istiod-ca-cert\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.797156 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.796971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-socket\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.804289 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.797047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz2zs\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-kube-api-access-sz2zs\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.804289 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.797512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-data\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.808371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.807351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-credential-socket\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.808371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.807422    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-podinfo\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.808371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.807470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-envoy\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.808371 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.807498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-certs\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908241 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc7a8bb-403d-4865-bf9d-94959070730b-istiod-ca-cert\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908241 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-socket\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908490 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz2zs\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-kube-api-access-sz2zs\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908490 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-data\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908490 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-credential-socket\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908669 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-podinfo\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908669 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-envoy\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908669 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-certs\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908858 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-socket\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908858 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-token\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.908858 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908810    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-data\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.909027 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.908855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-credential-socket\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.909118 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.909098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-certs\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.909218 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.909196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc7a8bb-403d-4865-bf9d-94959070730b-istiod-ca-cert\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.911231 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.911208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-envoy\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.911343 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.911267    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-podinfo\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.917714 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.917681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-token\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:26.917878 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:26.917861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz2zs\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-kube-api-access-sz2zs\") pod \"e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") " pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:27.018033 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.017940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:27.296779 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.296669    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"]
Sep 11 14:06:27.299848 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:06:27.299813    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc7a8bb_403d_4865_bf9d_94959070730b.slice/crio-12b7cf21b32839c73580dfa6b38215ab7f423524d35ecf8fb8fa534cccd67d06 WatchSource:0}: Error finding container 12b7cf21b32839c73580dfa6b38215ab7f423524d35ecf8fb8fa534cccd67d06: Status 404 returned error can't find the container with id 12b7cf21b32839c73580dfa6b38215ab7f423524d35ecf8fb8fa534cccd67d06
Sep 11 14:06:27.302715 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.302674    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:27.302980 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.302943    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:27.303110 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.303004    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:27.391514 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.391481    2563 generic.go:356] "Generic (PLEG): container finished" podID="a78f950e-299a-43fd-a1e8-7627b0f298dd" containerID="636c7521ec6dc6a356ba14eb441f21bee03f92fe4c85f45fd94a263b4a4f6b8a" exitCode=0
Sep 11 14:06:27.392038 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.391579    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx" event={"ID":"a78f950e-299a-43fd-a1e8-7627b0f298dd","Type":"ContainerDied","Data":"636c7521ec6dc6a356ba14eb441f21bee03f92fe4c85f45fd94a263b4a4f6b8a"}
Sep 11 14:06:27.393552 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.393525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" event={"ID":"1cc7a8bb-403d-4865-bf9d-94959070730b","Type":"ContainerStarted","Data":"cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27"}
Sep 11 14:06:27.393705 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:27.393569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" event={"ID":"1cc7a8bb-403d-4865-bf9d-94959070730b","Type":"ContainerStarted","Data":"12b7cf21b32839c73580dfa6b38215ab7f423524d35ecf8fb8fa534cccd67d06"}
Sep 11 14:06:28.018951 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.018917    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:28.024204 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.024178    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:28.045921 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.045872    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" podStartSLOduration=2.045853073 podStartE2EDuration="2.045853073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:27.429622035 +0000 UTC m=+1083.687822045" watchObservedRunningTime="2026-09-11 14:06:28.045853073 +0000 UTC m=+1084.304053081"
Sep 11 14:06:28.399940 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.399907    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:28.400918 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.400902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:28.551976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.551947    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"
Sep 11 14:06:28.624565 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.624523    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a78f950e-299a-43fd-a1e8-7627b0f298dd-maas-api-service-ca\") pod \"a78f950e-299a-43fd-a1e8-7627b0f298dd\" (UID: \"a78f950e-299a-43fd-a1e8-7627b0f298dd\") "
Sep 11 14:06:28.624943 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.624918    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78f950e-299a-43fd-a1e8-7627b0f298dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a78f950e-299a-43fd-a1e8-7627b0f298dd" (UID: "a78f950e-299a-43fd-a1e8-7627b0f298dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:06:28.726172 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:28.726077    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a78f950e-299a-43fd-a1e8-7627b0f298dd-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:06:29.408619 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:29.408588    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"
Sep 11 14:06:29.408619 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:29.408613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx" event={"ID":"a78f950e-299a-43fd-a1e8-7627b0f298dd","Type":"ContainerDied","Data":"b89ee8997ce1ee2c1727bc9105b88b7a6ee7a383bb036b0db90cc815b2631446"}
Sep 11 14:06:29.409167 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:29.408662    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b89ee8997ce1ee2c1727bc9105b88b7a6ee7a383bb036b0db90cc815b2631446"
Sep 11 14:06:35.444610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:35.444578    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:06:36.242366 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:36.242329    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"]
Sep 11 14:06:36.245085 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:36.245062    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffbdd858-m92cx"]
Sep 11 14:06:38.240460 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:38.240424    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a78f950e-299a-43fd-a1e8-7627b0f298dd" path="/var/lib/kubelet/pods/a78f950e-299a-43fd-a1e8-7627b0f298dd/volumes"
Sep 11 14:06:44.441976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:44.441925    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 14:06:44.442489 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:44.442004    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 14:06:44.934865 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:44.934832    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:06:46.516365 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:46.516328    2563 generic.go:356] "Generic (PLEG): container finished" podID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerID="cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27" exitCode=0
Sep 11 14:06:46.516920 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:46.516398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" event={"ID":"1cc7a8bb-403d-4865-bf9d-94959070730b","Type":"ContainerDied","Data":"cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27"}
Sep 11 14:06:46.516920 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:46.516846    2563 scope.go:117] "RemoveContainer" containerID="cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27"
Sep 11 14:06:46.517981 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:46.517951    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:46.518102 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:46.518018    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:46.518102 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:46.518049    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:47.529099 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:47.529059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" event={"ID":"1cc7a8bb-403d-4865-bf9d-94959070730b","Type":"ContainerStarted","Data":"c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5"}
Sep 11 14:06:48.019281 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:48.019244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:48.020547 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:48.019437    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 14:06:48.020811 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:48.020782    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 14:06:49.019281 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:49.019243    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 14:06:49.019933 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:49.019900    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 14:06:50.018948 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:50.018901    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 14:06:50.019175 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:50.019012    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 14:06:51.019226 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:51.019178    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 14:06:51.019905 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:51.019865    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 14:06:52.018788 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:52.018722    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 14:06:52.019096 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:52.019065    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 14:06:53.022270 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:53.022239    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:53.022699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:53.022525    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:06:53.023374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:06:53.023358    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:07:06.341720 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:06.341683    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:07:19.174536 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.174497    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"]
Sep 11 14:07:19.175096 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.175079    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78f950e-299a-43fd-a1e8-7627b0f298dd" containerName="revoke-keys"
Sep 11 14:07:19.175144 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.175099    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78f950e-299a-43fd-a1e8-7627b0f298dd" containerName="revoke-keys"
Sep 11 14:07:19.175230 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.175219    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78f950e-299a-43fd-a1e8-7627b0f298dd" containerName="revoke-keys"
Sep 11 14:07:19.179807 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.179782    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.182407 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.182381    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"openshift-service-ca.crt\""
Sep 11 14:07:19.183589 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.183536    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"auth-test-model-932c97-kserve-self-signed-certs\""
Sep 11 14:07:19.183713 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.183658    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"default-dockercfg-zrsfg\""
Sep 11 14:07:19.183795 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.183535    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"kube-root-ca.crt\""
Sep 11 14:07:19.187460 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.187423    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"]
Sep 11 14:07:19.226557 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.226525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-dshm\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.226741 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.226587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-model-cache\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.226741 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.226616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szz2j\" (UniqueName: \"kubernetes.io/projected/479424ad-3a30-4956-a828-94fe9c406ace-kube-api-access-szz2j\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.226741 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.226631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/479424ad-3a30-4956-a828-94fe9c406ace-tls-certs\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.226741 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.226651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-tmp-dir\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.226741 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.226718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-home\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.328320 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.328228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-model-cache\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.329250 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.329225    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szz2j\" (UniqueName: \"kubernetes.io/projected/479424ad-3a30-4956-a828-94fe9c406ace-kube-api-access-szz2j\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.329940 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.329907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/479424ad-3a30-4956-a828-94fe9c406ace-tls-certs\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.330165 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.329970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-tmp-dir\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.330165 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.330034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-home\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.330165 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.330089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-dshm\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.330775 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.329105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-model-cache\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.330775 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.330700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-tmp-dir\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.331177 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.330878    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-home\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.333252 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.333226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-dshm\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.334019 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.333988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/479424ad-3a30-4956-a828-94fe9c406ace-tls-certs\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.339452 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.339430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szz2j\" (UniqueName: \"kubernetes.io/projected/479424ad-3a30-4956-a828-94fe9c406ace-kube-api-access-szz2j\") pod \"auth-test-model-932c97-kserve-dc544f6d7-xkr7k\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") " pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.492564 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.492460    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:19.769048 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:19.768959    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"]
Sep 11 14:07:19.772973 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:07:19.772829    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod479424ad_3a30_4956_a828_94fe9c406ace.slice/crio-032f1c4f714fc885b2aa98033c43389dbac043e4943cff2bc2a60149e44863e1 WatchSource:0}: Error finding container 032f1c4f714fc885b2aa98033c43389dbac043e4943cff2bc2a60149e44863e1: Status 404 returned error can't find the container with id 032f1c4f714fc885b2aa98033c43389dbac043e4943cff2bc2a60149e44863e1
Sep 11 14:07:20.743571 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:20.743532    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k" event={"ID":"479424ad-3a30-4956-a828-94fe9c406ace","Type":"ContainerStarted","Data":"374bf050e191bb37b96272d108b66fd71f54f9af54de8cdd8c50de13baca24a8"}
Sep 11 14:07:20.743571 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:20.743578    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k" event={"ID":"479424ad-3a30-4956-a828-94fe9c406ace","Type":"ContainerStarted","Data":"032f1c4f714fc885b2aa98033c43389dbac043e4943cff2bc2a60149e44863e1"}
Sep 11 14:07:20.763151 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:20.763098    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k" podStartSLOduration=1.763078269 podStartE2EDuration="1.763078269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:07:20.759788981 +0000 UTC m=+1137.017988990" watchObservedRunningTime="2026-09-11 14:07:20.763078269 +0000 UTC m=+1137.021278278"
Sep 11 14:07:23.744883 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:23.744850    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:07:25.724365 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.724324    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"]
Sep 11 14:07:25.728430 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.728405    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.731689 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.731667    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-01faa9-openshift-default-dockercfg-bmk4l\""
Sep 11 14:07:25.740624 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.740597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"]
Sep 11 14:07:25.790429 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istiod-ca-cert\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.790599 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-socket\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.790599 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-credential-socket\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.790599 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-podinfo\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.790747 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ffsj\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-kube-api-access-2ffsj\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.790747 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-certs\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.790747 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-data\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.790747 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-envoy\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.790940 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.790826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-token\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.891685 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.891648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-envoy\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.891685 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.891697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-token\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.891723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istiod-ca-cert\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.891947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-socket\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892010 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.891991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-credential-socket\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892170 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-podinfo\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892170 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ffsj\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-kube-api-access-2ffsj\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892268 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-certs\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892268 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-data\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892373 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892302    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-socket\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892433 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892414    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-credential-socket\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892581 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-data\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892581 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-certs\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.892581 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.892554    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istiod-ca-cert\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.894311 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.894291    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-envoy\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.894812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.894793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-podinfo\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.901227 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.901204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-token\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:25.901360 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:25.901286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ffsj\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-kube-api-access-2ffsj\") pod \"e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") " pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:26.040126 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:26.040037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:26.323981 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:26.323950    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"]
Sep 11 14:07:26.325280 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:07:26.325254    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6382874f_b48e_4fd2_b7c4_c0745a559a7a.slice/crio-8ff66e982b41d8952ae09204877b7614768a4ccf79235d878dacc7ceaa25bb1e WatchSource:0}: Error finding container 8ff66e982b41d8952ae09204877b7614768a4ccf79235d878dacc7ceaa25bb1e: Status 404 returned error can't find the container with id 8ff66e982b41d8952ae09204877b7614768a4ccf79235d878dacc7ceaa25bb1e
Sep 11 14:07:26.327702 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:26.327669    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:26.327840 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:26.327738    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:26.327840 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:26.327787    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:26.782183 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:26.782138    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" event={"ID":"6382874f-b48e-4fd2-b7c4-c0745a559a7a","Type":"ContainerStarted","Data":"fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d"}
Sep 11 14:07:26.782607 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:26.782206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" event={"ID":"6382874f-b48e-4fd2-b7c4-c0745a559a7a","Type":"ContainerStarted","Data":"8ff66e982b41d8952ae09204877b7614768a4ccf79235d878dacc7ceaa25bb1e"}
Sep 11 14:07:26.804693 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:26.804643    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" podStartSLOduration=1.804625106 podStartE2EDuration="1.804625106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:07:26.803310206 +0000 UTC m=+1143.061510216" watchObservedRunningTime="2026-09-11 14:07:26.804625106 +0000 UTC m=+1143.062825112"
Sep 11 14:07:27.040563 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:27.040461    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:27.042086 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:27.042061    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 11 14:07:27.042171 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:27.042115    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 11 14:07:28.044897 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:28.044864    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:28.794740 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:28.794709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:28.795745 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:28.795726    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:29.493631 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:29.493590    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:29.494161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:29.493649    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:29.508248 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:29.508218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:29.819309 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:29.819226    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:07:37.539192 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:37.539141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:07:47.928415 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:47.928376    2563 generic.go:356] "Generic (PLEG): container finished" podID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerID="fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d" exitCode=0
Sep 11 14:07:47.928915 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:47.928454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" event={"ID":"6382874f-b48e-4fd2-b7c4-c0745a559a7a","Type":"ContainerDied","Data":"fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d"}
Sep 11 14:07:47.928968 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:47.928926    2563 scope.go:117] "RemoveContainer" containerID="fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d"
Sep 11 14:07:47.930038 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:47.930004    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:47.930148 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:47.930096    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:47.930226 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:47.930146    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:48.936840 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:48.936807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" event={"ID":"6382874f-b48e-4fd2-b7c4-c0745a559a7a","Type":"ContainerStarted","Data":"87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7"}
Sep 11 14:07:49.041226 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:49.041193    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:49.041450 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:49.041430    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 11 14:07:49.041507 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:49.041476    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 11 14:07:50.041233 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:50.041194    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 11 14:07:50.041722 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:50.041253    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 11 14:07:51.041167 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.041127    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 11 14:07:51.041431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.041210    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 11 14:07:51.572317 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.572279    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"]
Sep 11 14:07:51.576311 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.576286    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.578976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.578944    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"openshift-service-ca.crt\""
Sep 11 14:07:51.579140 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.579025    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"kube-root-ca.crt\""
Sep 11 14:07:51.580388 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.580367    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"auth-test-model-367517-kserve-self-signed-certs\""
Sep 11 14:07:51.580498 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.580397    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"default-dockercfg-gwmlw\""
Sep 11 14:07:51.584849 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.584818    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"]
Sep 11 14:07:51.759341 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.759302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-model-cache\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.759341 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.759346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tmp-dir\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.759619 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.759367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tls-certs\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.759619 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.759396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-dshm\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.759619 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.759475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-home\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.759619 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.759520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n262z\" (UniqueName: \"kubernetes.io/projected/477a47eb-01d8-4c4c-abd7-16cf44f9508a-kube-api-access-n262z\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860152 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-home\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n262z\" (UniqueName: \"kubernetes.io/projected/477a47eb-01d8-4c4c-abd7-16cf44f9508a-kube-api-access-n262z\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-model-cache\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tmp-dir\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tls-certs\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-dshm\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860646 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-home\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860702 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-model-cache\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.860793 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.860691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tmp-dir\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.862813 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.862789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-dshm\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.863061 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.863045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tls-certs\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.870296 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.870270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n262z\" (UniqueName: \"kubernetes.io/projected/477a47eb-01d8-4c4c-abd7-16cf44f9508a-kube-api-access-n262z\") pod \"auth-test-model-367517-kserve-7545c8c695-ss897\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") " pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:51.887428 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:51.887396    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:07:52.041411 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:52.041368    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 11 14:07:52.041605 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:52.041440    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 11 14:07:52.191643 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:52.191616    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"]
Sep 11 14:07:52.193634 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:07:52.193603    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod477a47eb_01d8_4c4c_abd7_16cf44f9508a.slice/crio-339bacf0e9cdd8395c9d0dcfc7af0131073d6b33a9ccdf1028651e0bb036049a WatchSource:0}: Error finding container 339bacf0e9cdd8395c9d0dcfc7af0131073d6b33a9ccdf1028651e0bb036049a: Status 404 returned error can't find the container with id 339bacf0e9cdd8395c9d0dcfc7af0131073d6b33a9ccdf1028651e0bb036049a
Sep 11 14:07:52.967428 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:52.967386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897" event={"ID":"477a47eb-01d8-4c4c-abd7-16cf44f9508a","Type":"ContainerStarted","Data":"54fc17f4a8564dd6be5c5f76fb61038190434a5db80cb83e10f6c696e38e932e"}
Sep 11 14:07:52.967428 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:52.967435    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897" event={"ID":"477a47eb-01d8-4c4c-abd7-16cf44f9508a","Type":"ContainerStarted","Data":"339bacf0e9cdd8395c9d0dcfc7af0131073d6b33a9ccdf1028651e0bb036049a"}
Sep 11 14:07:52.985845 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:52.985788    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897" podStartSLOduration=1.9857717799999999 podStartE2EDuration="1.98577178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:07:52.983353683 +0000 UTC m=+1169.241553692" watchObservedRunningTime="2026-09-11 14:07:52.98577178 +0000 UTC m=+1169.243971786"
Sep 11 14:07:53.040867 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:53.040822    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 11 14:07:53.041083 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:53.040893    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 11 14:07:54.133878 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:54.133835    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:54.134384 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:54.134279    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:54.137353 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:54.137329    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:07:55.846103 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:07:55.846067    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:08:00.749823 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:00.749776    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7540-37c427"]
Sep 11 14:08:00.754668 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:00.754642    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7540-37c427"
Sep 11 14:08:00.757890 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:00.757862    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2qcb7\""
Sep 11 14:08:00.760866 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:00.760840    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7540-37c427"]
Sep 11 14:08:00.848853 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:00.848807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh5pj\" (UniqueName: \"kubernetes.io/projected/9d01306f-812a-4773-bbcb-95e01fcd29b8-kube-api-access-bh5pj\") pod \"test-curl-7540-37c427\" (UID: \"9d01306f-812a-4773-bbcb-95e01fcd29b8\") " pod="opendatahub/test-curl-7540-37c427"
Sep 11 14:08:00.950207 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:00.950170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5pj\" (UniqueName: \"kubernetes.io/projected/9d01306f-812a-4773-bbcb-95e01fcd29b8-kube-api-access-bh5pj\") pod \"test-curl-7540-37c427\" (UID: \"9d01306f-812a-4773-bbcb-95e01fcd29b8\") " pod="opendatahub/test-curl-7540-37c427"
Sep 11 14:08:00.959154 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:00.959121    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5pj\" (UniqueName: \"kubernetes.io/projected/9d01306f-812a-4773-bbcb-95e01fcd29b8-kube-api-access-bh5pj\") pod \"test-curl-7540-37c427\" (UID: \"9d01306f-812a-4773-bbcb-95e01fcd29b8\") " pod="opendatahub/test-curl-7540-37c427"
Sep 11 14:08:01.065986 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:01.065884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7540-37c427"
Sep 11 14:08:01.342842 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:01.342818    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7540-37c427"]
Sep 11 14:08:01.348436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:01.348416    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:08:01.888365 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:01.888326    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:08:01.888904 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:01.888389    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:08:01.901078 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:01.901050    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:08:02.023224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:02.023188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7540-37c427" event={"ID":"9d01306f-812a-4773-bbcb-95e01fcd29b8","Type":"ContainerStarted","Data":"01828a09fce56b905cdad47884f6885ad36f82473b63cb3caaf6146512f1c3c4"}
Sep 11 14:08:02.034809 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:02.034780    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:08:03.031081 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:03.031046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7540-37c427" event={"ID":"9d01306f-812a-4773-bbcb-95e01fcd29b8","Type":"ContainerStarted","Data":"8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268"}
Sep 11 14:08:03.046394 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:03.046340    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7540-37c427" podStartSLOduration=2.213199061 podStartE2EDuration="3.046322495s" totalImagesPullingTime="833.123434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:08:03.045846891 +0000 UTC m=+1179.304046911" watchObservedRunningTime="2026-09-11 14:08:03.046322495 +0000 UTC m=+1179.304522549"
Sep 11 14:08:03.673283 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:03.673243    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7540-37c427"]
Sep 11 14:08:03.676742 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:03.676704    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7540-37c427"]
Sep 11 14:08:04.037827 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:08:04.037794    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7540-37c427" container="test-curl-7540-37c427"
Sep 11 14:08:04.567810 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.567775    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"]
Sep 11 14:08:04.568201 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.568188    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d01306f-812a-4773-bbcb-95e01fcd29b8" containerName="test-curl-7540-37c427"
Sep 11 14:08:04.568258 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.568202    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d01306f-812a-4773-bbcb-95e01fcd29b8" containerName="test-curl-7540-37c427"
Sep 11 14:08:04.568301 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.568269    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d01306f-812a-4773-bbcb-95e01fcd29b8" containerName="test-curl-7540-37c427"
Sep 11 14:08:04.571718 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.571667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.575244 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.575221    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fc83333c-openshift-default-dockercfg-q8pfh\""
Sep 11 14:08:04.584802 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.584742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-podinfo\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.585061 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.585021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-token\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.585061 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.585058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2vzf\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-kube-api-access-n2vzf\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.585241 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.585174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6806a5b-5d97-4e09-a052-ed7c37251727-istiod-ca-cert\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.585241 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.585235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-socket\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.585355 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.585292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-data\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.585355 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.585328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-certs\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.585355 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.585350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-credential-socket\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.585531 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.585394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-envoy\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.588373 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.588347    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"]
Sep 11 14:08:04.686509 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-podinfo\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.686727 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-token\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.686727 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2vzf\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-kube-api-access-n2vzf\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.686727 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6806a5b-5d97-4e09-a052-ed7c37251727-istiod-ca-cert\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.686727 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-socket\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.686727 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-data\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.687049 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-certs\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.687049 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-credential-socket\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.687049 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.686930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-envoy\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.688223 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.688189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6806a5b-5d97-4e09-a052-ed7c37251727-istiod-ca-cert\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.689419 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.689387    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-socket\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.689419 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.689408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-data\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.689582 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.689514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-certs\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.689919 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.689898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-envoy\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.690066 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.689969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-credential-socket\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.690066 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.689920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-podinfo\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.696853 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.696827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2vzf\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-kube-api-access-n2vzf\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.697288 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.697264    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-token\") pod \"e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") " pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:04.882844 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:04.882802    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:05.044244 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:05.044185    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7540-37c427" podUID="9d01306f-812a-4773-bbcb-95e01fcd29b8" containerName="test-curl-7540-37c427" containerID="cri-o://8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268" gracePeriod=2
Sep 11 14:08:05.170394 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:05.170368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"]
Sep 11 14:08:05.172372 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:08:05.172340    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6806a5b_5d97_4e09_a052_ed7c37251727.slice/crio-11ea0f8ce6e175539b3d0956dcb9374ebb82ab81e8e3a163631c9101875bf48a WatchSource:0}: Error finding container 11ea0f8ce6e175539b3d0956dcb9374ebb82ab81e8e3a163631c9101875bf48a: Status 404 returned error can't find the container with id 11ea0f8ce6e175539b3d0956dcb9374ebb82ab81e8e3a163631c9101875bf48a
Sep 11 14:08:05.174912 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:05.174872    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:08:05.175048 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:05.174956    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:08:05.175048 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:05.175000    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:08:06.052571 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:06.052537    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" event={"ID":"d6806a5b-5d97-4e09-a052-ed7c37251727","Type":"ContainerStarted","Data":"76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d"}
Sep 11 14:08:06.053059 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:06.052586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" event={"ID":"d6806a5b-5d97-4e09-a052-ed7c37251727","Type":"ContainerStarted","Data":"11ea0f8ce6e175539b3d0956dcb9374ebb82ab81e8e3a163631c9101875bf48a"}
Sep 11 14:08:06.076875 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:06.076814    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" podStartSLOduration=2.076795213 podStartE2EDuration="2.076795213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:08:06.073488893 +0000 UTC m=+1182.331688904" watchObservedRunningTime="2026-09-11 14:08:06.076795213 +0000 UTC m=+1182.334995223"
Sep 11 14:08:06.883237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:06.883196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:06.888771 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:06.888727    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:07.059142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:07.059120    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:07.060108 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:07.060089    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:07.238728 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:07.238698    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7540-37c427"
Sep 11 14:08:07.312604 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:07.312572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh5pj\" (UniqueName: \"kubernetes.io/projected/9d01306f-812a-4773-bbcb-95e01fcd29b8-kube-api-access-bh5pj\") pod \"9d01306f-812a-4773-bbcb-95e01fcd29b8\" (UID: \"9d01306f-812a-4773-bbcb-95e01fcd29b8\") "
Sep 11 14:08:07.314969 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:07.314942    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d01306f-812a-4773-bbcb-95e01fcd29b8-kube-api-access-bh5pj" (OuterVolumeSpecName: "kube-api-access-bh5pj") pod "9d01306f-812a-4773-bbcb-95e01fcd29b8" (UID: "9d01306f-812a-4773-bbcb-95e01fcd29b8"). InnerVolumeSpecName "kube-api-access-bh5pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:08:07.414223 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:07.414145    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh5pj\" (UniqueName: \"kubernetes.io/projected/9d01306f-812a-4773-bbcb-95e01fcd29b8-kube-api-access-bh5pj\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:08:08.068271 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:08.068234    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d01306f-812a-4773-bbcb-95e01fcd29b8" containerID="8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268" exitCode=137
Sep 11 14:08:08.068804 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:08.068288    2563 scope.go:117] "RemoveContainer" containerID="8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268"
Sep 11 14:08:08.068804 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:08.068338    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7540-37c427"
Sep 11 14:08:08.078904 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:08.078883    2563 scope.go:117] "RemoveContainer" containerID="8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268"
Sep 11 14:08:08.079227 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:08:08.079208    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268\": container with ID starting with 8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268 not found: ID does not exist" containerID="8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268"
Sep 11 14:08:08.079281 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:08.079237    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268"} err="failed to get container status \"8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268\": rpc error: code = NotFound desc = could not find container \"8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268\": container with ID starting with 8cd302e0e418dfc2ca1123781c0f231631a87147357a36ba10f7ab9627952268 not found: ID does not exist"
Sep 11 14:08:08.079281 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:08:08.079242    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7540-37c427\" is forbidden: User \"system:node:ip-10-0-137-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-192.ec2.internal' and this object" podUID="9d01306f-812a-4773-bbcb-95e01fcd29b8" pod="opendatahub/test-curl-7540-37c427"
Sep 11 14:08:08.244026 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:08.243982    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d01306f-812a-4773-bbcb-95e01fcd29b8" path="/var/lib/kubelet/pods/9d01306f-812a-4773-bbcb-95e01fcd29b8/volumes"
Sep 11 14:08:18.736289 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:18.736250    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:08:24.358287 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:24.358252    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:08:24.362393 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:24.362367    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:08:30.209871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:30.209829    2563 generic.go:356] "Generic (PLEG): container finished" podID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerID="76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d" exitCode=0
Sep 11 14:08:30.210437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:30.209907    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" event={"ID":"d6806a5b-5d97-4e09-a052-ed7c37251727","Type":"ContainerDied","Data":"76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d"}
Sep 11 14:08:30.210510 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:30.210461    2563 scope.go:117] "RemoveContainer" containerID="76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d"
Sep 11 14:08:30.211638 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:30.211610    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:08:30.211776 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:30.211693    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:08:30.211849 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:30.211816    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:08:31.225000 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:31.224951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" event={"ID":"d6806a5b-5d97-4e09-a052-ed7c37251727","Type":"ContainerStarted","Data":"c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe"}
Sep 11 14:08:31.883715 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:31.883678    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:31.884025 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:31.884001    2563 patch_prober.go:28] interesting pod/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 14:08:31.884087 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:31.884063    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 14:08:32.883555 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:32.883510    2563 patch_prober.go:28] interesting pod/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 14:08:32.884079 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:32.883576    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 14:08:33.884203 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:33.884162    2563 patch_prober.go:28] interesting pod/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 14:08:33.884699 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:33.884233    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 14:08:34.884177 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:34.884125    2563 patch_prober.go:28] interesting pod/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 14:08:34.884400 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:34.884208    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 14:08:35.884217 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:35.884178    2563 patch_prober.go:28] interesting pod/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 14:08:35.884409 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:35.884251    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 14:08:36.887408 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.887377    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:36.887833 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.887669    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:36.888455 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.888433    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:08:36.972443 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.972397    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"]
Sep 11 14:08:36.976351 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.976324    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:36.979338 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.979307    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fc83333c\"/\"openshift-service-ca.crt\""
Sep 11 14:08:36.979338 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.979336    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fc83333c\"/\"kube-root-ca.crt\""
Sep 11 14:08:36.979575 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.979339    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fc83333c\"/\"e2e-lifecycle-model-fc83333c-kserve-self-signed-certs\""
Sep 11 14:08:36.979642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.979593    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fc83333c\"/\"default-dockercfg-df4l9\""
Sep 11 14:08:36.990538 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:36.990507    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"]
Sep 11 14:08:37.122028 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.121979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-home\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.122220 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.122050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202f8943-c9c4-41b6-8324-ae820fb1fef9-tls-certs\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.122220 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.122108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-tmp-dir\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.122220 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.122159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-model-cache\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.122220 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.122205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2jss\" (UniqueName: \"kubernetes.io/projected/202f8943-c9c4-41b6-8324-ae820fb1fef9-kube-api-access-k2jss\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.122358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.122225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-dshm\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.223718 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.223614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-model-cache\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.223718 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.223709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2jss\" (UniqueName: \"kubernetes.io/projected/202f8943-c9c4-41b6-8324-ae820fb1fef9-kube-api-access-k2jss\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.223972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.223738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-dshm\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.223972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.223827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-home\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.223972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.223866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202f8943-c9c4-41b6-8324-ae820fb1fef9-tls-certs\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.223972 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.223949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-tmp-dir\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.224188 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.224125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-model-cache\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.224188 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.224173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-home\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.224282 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.224248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-tmp-dir\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.226325 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.226301    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-dshm\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.226502 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.226485    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202f8943-c9c4-41b6-8324-ae820fb1fef9-tls-certs\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.232070 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.232049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2jss\" (UniqueName: \"kubernetes.io/projected/202f8943-c9c4-41b6-8324-ae820fb1fef9-kube-api-access-k2jss\") pod \"e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") " pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.291788 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.291695    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:37.335782 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.335728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:08:37.570421 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:37.570386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"]
Sep 11 14:08:37.571916 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:08:37.571886    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod202f8943_c9c4_41b6_8324_ae820fb1fef9.slice/crio-58f555190ce89b899ec89aea24b68f7ab1fa55f6bce74fab0b712d5c824feb81 WatchSource:0}: Error finding container 58f555190ce89b899ec89aea24b68f7ab1fa55f6bce74fab0b712d5c824feb81: Status 404 returned error can't find the container with id 58f555190ce89b899ec89aea24b68f7ab1fa55f6bce74fab0b712d5c824feb81
Sep 11 14:08:38.283823 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:38.283791    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m" event={"ID":"202f8943-c9c4-41b6-8324-ae820fb1fef9","Type":"ContainerStarted","Data":"a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467"}
Sep 11 14:08:38.284248 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:38.283832    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m" event={"ID":"202f8943-c9c4-41b6-8324-ae820fb1fef9","Type":"ContainerStarted","Data":"58f555190ce89b899ec89aea24b68f7ab1fa55f6bce74fab0b712d5c824feb81"}
Sep 11 14:08:38.305428 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:38.305352    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m" podStartSLOduration=2.305326244 podStartE2EDuration="2.305326244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:08:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:08:38.302355456 +0000 UTC m=+1214.560555477" watchObservedRunningTime="2026-09-11 14:08:38.305326244 +0000 UTC m=+1214.563526253"
Sep 11 14:08:47.292648 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:47.292588    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:47.293161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:47.292689    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:47.305818 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:47.305787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:47.351288 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:47.351252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:08:54.335481 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:08:54.335446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:09:02.109307 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.109265    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"]
Sep 11 14:09:02.113111 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.113090    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"
Sep 11 14:09:02.116136 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.116101    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:09:02.116136 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.116123    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:09:02.132553 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.132520    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"]
Sep 11 14:09:02.188828 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.188789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ce7f67-7e7b-4065-9851-20ec08df28c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj\" (UID: \"b3ce7f67-7e7b-4065-9851-20ec08df28c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"
Sep 11 14:09:02.289540 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.289504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ce7f67-7e7b-4065-9851-20ec08df28c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj\" (UID: \"b3ce7f67-7e7b-4065-9851-20ec08df28c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"
Sep 11 14:09:02.290199 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.290177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ce7f67-7e7b-4065-9851-20ec08df28c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj\" (UID: \"b3ce7f67-7e7b-4065-9851-20ec08df28c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"
Sep 11 14:09:02.424658 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.424554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"
Sep 11 14:09:02.702637 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:02.702612    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"]
Sep 11 14:09:02.704518 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:09:02.704488    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ce7f67_7e7b_4065_9851_20ec08df28c8.slice/crio-b4c8e0a5fd6769467f69bf466c1ee4afbc669b935e167861c8bf659f12939555 WatchSource:0}: Error finding container b4c8e0a5fd6769467f69bf466c1ee4afbc669b935e167861c8bf659f12939555: Status 404 returned error can't find the container with id b4c8e0a5fd6769467f69bf466c1ee4afbc669b935e167861c8bf659f12939555
Sep 11 14:09:03.444989 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:03.444953    2563 generic.go:356] "Generic (PLEG): container finished" podID="b3ce7f67-7e7b-4065-9851-20ec08df28c8" containerID="6b6a48137fd34a2713575b5da9f0cbbdb14de67779e892b8a7f094536b0ac171" exitCode=0
Sep 11 14:09:03.445390 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:03.445045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj" event={"ID":"b3ce7f67-7e7b-4065-9851-20ec08df28c8","Type":"ContainerDied","Data":"6b6a48137fd34a2713575b5da9f0cbbdb14de67779e892b8a7f094536b0ac171"}
Sep 11 14:09:03.445390 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:03.445131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj" event={"ID":"b3ce7f67-7e7b-4065-9851-20ec08df28c8","Type":"ContainerStarted","Data":"b4c8e0a5fd6769467f69bf466c1ee4afbc669b935e167861c8bf659f12939555"}
Sep 11 14:09:04.596809 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:04.596780    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"
Sep 11 14:09:04.715139 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:04.715045    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ce7f67-7e7b-4065-9851-20ec08df28c8-maas-api-service-ca\") pod \"b3ce7f67-7e7b-4065-9851-20ec08df28c8\" (UID: \"b3ce7f67-7e7b-4065-9851-20ec08df28c8\") "
Sep 11 14:09:04.715423 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:04.715397    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ce7f67-7e7b-4065-9851-20ec08df28c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3ce7f67-7e7b-4065-9851-20ec08df28c8" (UID: "b3ce7f67-7e7b-4065-9851-20ec08df28c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:09:04.816055 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:04.816020    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ce7f67-7e7b-4065-9851-20ec08df28c8-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:05.458856 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:05.458829    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"
Sep 11 14:09:05.459039 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:05.458822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj" event={"ID":"b3ce7f67-7e7b-4065-9851-20ec08df28c8","Type":"ContainerDied","Data":"b4c8e0a5fd6769467f69bf466c1ee4afbc669b935e167861c8bf659f12939555"}
Sep 11 14:09:05.459039 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:05.458949    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4c8e0a5fd6769467f69bf466c1ee4afbc669b935e167861c8bf659f12939555"
Sep 11 14:09:13.446191 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:13.446150    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:09:17.876918 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:17.876832    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"]
Sep 11 14:09:17.881263 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:17.881232    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fc83333c-s4vhj"]
Sep 11 14:09:18.241506 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:18.241418    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3ce7f67-7e7b-4065-9851-20ec08df28c8" path="/var/lib/kubelet/pods/b3ce7f67-7e7b-4065-9851-20ec08df28c8/volumes"
Sep 11 14:09:24.394998 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:24.394965    2563 scope.go:117] "RemoveContainer" containerID="b699e3718cf52f9e8dfa6e4bd591ecc92b85df9da22c274043edb17dfe6a68db"
Sep 11 14:09:24.404938 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:24.404909    2563 scope.go:117] "RemoveContainer" containerID="9592f965fe5898043d74c6e6b803faae295b8a260b86e470f5bb990a6f05d054"
Sep 11 14:09:26.051501 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:26.051465    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"]
Sep 11 14:09:31.936562 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:31.936524    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:09:41.071887 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.071843    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m" podUID="202f8943-c9c4-41b6-8324-ae820fb1fef9" containerName="main" containerID="cri-o://a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467" gracePeriod=45
Sep 11 14:09:41.336986 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.336917    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:09:41.483904 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.483853    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-home\") pod \"202f8943-c9c4-41b6-8324-ae820fb1fef9\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") "
Sep 11 14:09:41.483904 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.483908    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-dshm\") pod \"202f8943-c9c4-41b6-8324-ae820fb1fef9\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") "
Sep 11 14:09:41.484245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.483936    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202f8943-c9c4-41b6-8324-ae820fb1fef9-tls-certs\") pod \"202f8943-c9c4-41b6-8324-ae820fb1fef9\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") "
Sep 11 14:09:41.484245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484003    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2jss\" (UniqueName: \"kubernetes.io/projected/202f8943-c9c4-41b6-8324-ae820fb1fef9-kube-api-access-k2jss\") pod \"202f8943-c9c4-41b6-8324-ae820fb1fef9\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") "
Sep 11 14:09:41.484245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484058    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-tmp-dir\") pod \"202f8943-c9c4-41b6-8324-ae820fb1fef9\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") "
Sep 11 14:09:41.484245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484094    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-model-cache\") pod \"202f8943-c9c4-41b6-8324-ae820fb1fef9\" (UID: \"202f8943-c9c4-41b6-8324-ae820fb1fef9\") "
Sep 11 14:09:41.484245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484153    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-home" (OuterVolumeSpecName: "home") pod "202f8943-c9c4-41b6-8324-ae820fb1fef9" (UID: "202f8943-c9c4-41b6-8324-ae820fb1fef9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:41.484511 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484286    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "202f8943-c9c4-41b6-8324-ae820fb1fef9" (UID: "202f8943-c9c4-41b6-8324-ae820fb1fef9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:41.484511 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484361    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-model-cache" (OuterVolumeSpecName: "model-cache") pod "202f8943-c9c4-41b6-8324-ae820fb1fef9" (UID: "202f8943-c9c4-41b6-8324-ae820fb1fef9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:41.484511 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484497    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:41.484511 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484510    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:41.484924 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.484518    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:41.486385 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.486363    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-dshm" (OuterVolumeSpecName: "dshm") pod "202f8943-c9c4-41b6-8324-ae820fb1fef9" (UID: "202f8943-c9c4-41b6-8324-ae820fb1fef9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:41.486493 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.486415    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/202f8943-c9c4-41b6-8324-ae820fb1fef9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "202f8943-c9c4-41b6-8324-ae820fb1fef9" (UID: "202f8943-c9c4-41b6-8324-ae820fb1fef9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:41.486493 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.486480    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202f8943-c9c4-41b6-8324-ae820fb1fef9-kube-api-access-k2jss" (OuterVolumeSpecName: "kube-api-access-k2jss") pod "202f8943-c9c4-41b6-8324-ae820fb1fef9" (UID: "202f8943-c9c4-41b6-8324-ae820fb1fef9"). InnerVolumeSpecName "kube-api-access-k2jss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:09:41.585361 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.585311    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202f8943-c9c4-41b6-8324-ae820fb1fef9-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:41.585361 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.585359    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202f8943-c9c4-41b6-8324-ae820fb1fef9-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:41.585361 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.585370    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2jss\" (UniqueName: \"kubernetes.io/projected/202f8943-c9c4-41b6-8324-ae820fb1fef9-kube-api-access-k2jss\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:41.700206 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.700168    2563 generic.go:356] "Generic (PLEG): container finished" podID="202f8943-c9c4-41b6-8324-ae820fb1fef9" containerID="a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467" exitCode=0
Sep 11 14:09:41.700402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.700217    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"
Sep 11 14:09:41.700402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.700221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m" event={"ID":"202f8943-c9c4-41b6-8324-ae820fb1fef9","Type":"ContainerDied","Data":"a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467"}
Sep 11 14:09:41.700402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.700275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m" event={"ID":"202f8943-c9c4-41b6-8324-ae820fb1fef9","Type":"ContainerDied","Data":"58f555190ce89b899ec89aea24b68f7ab1fa55f6bce74fab0b712d5c824feb81"}
Sep 11 14:09:41.700402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.700297    2563 scope.go:117] "RemoveContainer" containerID="a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467"
Sep 11 14:09:41.711065 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.711039    2563 scope.go:117] "RemoveContainer" containerID="a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467"
Sep 11 14:09:41.711442 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:09:41.711419    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467\": container with ID starting with a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467 not found: ID does not exist" containerID="a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467"
Sep 11 14:09:41.711555 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.711450    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467"} err="failed to get container status \"a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467\": rpc error: code = NotFound desc = could not find container \"a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467\": container with ID starting with a008a0df910ab346556e81380af372fee3a305876ddadc568a366b246b13f467 not found: ID does not exist"
Sep 11 14:09:41.722717 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.722680    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"]
Sep 11 14:09:41.729737 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:41.729690    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fc83333c/e2e-lifecycle-model-fc83333c-kserve-76c68965d7-6tp4m"]
Sep 11 14:09:42.242029 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:42.241996    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="202f8943-c9c4-41b6-8324-ae820fb1fef9" path="/var/lib/kubelet/pods/202f8943-c9c4-41b6-8324-ae820fb1fef9/volumes"
Sep 11 14:09:48.941024 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:48.940990    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:09:52.935058 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:52.935017    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"]
Sep 11 14:09:52.935496 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:52.935290    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy" containerID="cri-o://c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe" gracePeriod=30
Sep 11 14:09:58.206559 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.206485    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:09:58.272540 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272501    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-credential-socket\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.272731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272555    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-data\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.272731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-envoy\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.272731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272605    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6806a5b-5d97-4e09-a052-ed7c37251727-istiod-ca-cert\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.272731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272640    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-socket\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.272731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272696    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-podinfo\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.273030 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272791    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-certs\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.273030 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272834    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-token\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.273030 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272862    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2vzf\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-kube-api-access-n2vzf\") pod \"d6806a5b-5d97-4e09-a052-ed7c37251727\" (UID: \"d6806a5b-5d97-4e09-a052-ed7c37251727\") "
Sep 11 14:09:58.273030 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272891    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:58.273030 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.272906    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:58.273266 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.273121    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:58.273266 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.273161    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6806a5b-5d97-4e09-a052-ed7c37251727-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:09:58.273266 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.273165    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:58.273375 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.273267    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.273375 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.273287    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.273375 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.273300    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.273375 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.273313    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6806a5b-5d97-4e09-a052-ed7c37251727-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.273375 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.273326    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.275107 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.275073    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:09:58.275241 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.275160    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-kube-api-access-n2vzf" (OuterVolumeSpecName: "kube-api-access-n2vzf") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "kube-api-access-n2vzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:09:58.275543 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.275514    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:09:58.275543 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.275525    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6806a5b-5d97-4e09-a052-ed7c37251727" (UID: "d6806a5b-5d97-4e09-a052-ed7c37251727"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:09:58.374296 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.374257    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.374296 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.374293    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2vzf\" (UniqueName: \"kubernetes.io/projected/d6806a5b-5d97-4e09-a052-ed7c37251727-kube-api-access-n2vzf\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.374296 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.374303    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.374550 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.374312    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6806a5b-5d97-4e09-a052-ed7c37251727-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:09:58.817332 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.817292    2563 generic.go:356] "Generic (PLEG): container finished" podID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerID="c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe" exitCode=0
Sep 11 14:09:58.817523 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.817341    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"
Sep 11 14:09:58.817523 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.817366    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" event={"ID":"d6806a5b-5d97-4e09-a052-ed7c37251727","Type":"ContainerDied","Data":"c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe"}
Sep 11 14:09:58.817523 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.817414    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt" event={"ID":"d6806a5b-5d97-4e09-a052-ed7c37251727","Type":"ContainerDied","Data":"11ea0f8ce6e175539b3d0956dcb9374ebb82ab81e8e3a163631c9101875bf48a"}
Sep 11 14:09:58.817523 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.817438    2563 scope.go:117] "RemoveContainer" containerID="c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe"
Sep 11 14:09:58.828456 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.828436    2563 scope.go:117] "RemoveContainer" containerID="76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d"
Sep 11 14:09:58.841095 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.841070    2563 scope.go:117] "RemoveContainer" containerID="c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe"
Sep 11 14:09:58.841769 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.841569    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"]
Sep 11 14:09:58.841769 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:09:58.841611    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe\": container with ID starting with c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe not found: ID does not exist" containerID="c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe"
Sep 11 14:09:58.841769 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.841713    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe"} err="failed to get container status \"c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe\": rpc error: code = NotFound desc = could not find container \"c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe\": container with ID starting with c31385a376d1af19d95423dacf4cf741e904f6880592226d062c01c81360dffe not found: ID does not exist"
Sep 11 14:09:58.841769 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.841769    2563 scope.go:117] "RemoveContainer" containerID="76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d"
Sep 11 14:09:58.842246 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:09:58.842132    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d\": container with ID starting with 76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d not found: ID does not exist" containerID="76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d"
Sep 11 14:09:58.842246 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.842164    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d"} err="failed to get container status \"76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d\": rpc error: code = NotFound desc = could not find container \"76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d\": container with ID starting with 76d6c1f6b5bd3a4fecc8e8eec335966b5399f34b9b549eb332244ffd034e838d not found: ID does not exist"
Sep 11 14:09:58.845377 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:09:58.845353    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fc83333c-openshift-default-6fdd4f659b-cx5wt"]
Sep 11 14:10:00.247468 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:00.247431    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" path="/var/lib/kubelet/pods/d6806a5b-5d97-4e09-a052-ed7c37251727/volumes"
Sep 11 14:10:03.704714 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:03.704677    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"]
Sep 11 14:10:04.751828 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:04.751782    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"]
Sep 11 14:10:07.946733 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:07.946684    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:10:18.725013 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:18.724938    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k" podUID="479424ad-3a30-4956-a828-94fe9c406ace" containerName="main" containerID="cri-o://374bf050e191bb37b96272d108b66fd71f54f9af54de8cdd8c50de13baca24a8" gracePeriod=45
Sep 11 14:10:18.954542 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:18.954502    2563 generic.go:356] "Generic (PLEG): container finished" podID="479424ad-3a30-4956-a828-94fe9c406ace" containerID="374bf050e191bb37b96272d108b66fd71f54f9af54de8cdd8c50de13baca24a8" exitCode=0
Sep 11 14:10:18.954700 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:18.954569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k" event={"ID":"479424ad-3a30-4956-a828-94fe9c406ace","Type":"ContainerDied","Data":"374bf050e191bb37b96272d108b66fd71f54f9af54de8cdd8c50de13baca24a8"}
Sep 11 14:10:18.974446 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:18.974422    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:10:19.084194 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084087    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-dshm\") pod \"479424ad-3a30-4956-a828-94fe9c406ace\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") "
Sep 11 14:10:19.084194 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084135    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/479424ad-3a30-4956-a828-94fe9c406ace-tls-certs\") pod \"479424ad-3a30-4956-a828-94fe9c406ace\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") "
Sep 11 14:10:19.084444 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084281    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szz2j\" (UniqueName: \"kubernetes.io/projected/479424ad-3a30-4956-a828-94fe9c406ace-kube-api-access-szz2j\") pod \"479424ad-3a30-4956-a828-94fe9c406ace\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") "
Sep 11 14:10:19.084444 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084315    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-tmp-dir\") pod \"479424ad-3a30-4956-a828-94fe9c406ace\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") "
Sep 11 14:10:19.084444 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084348    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-home\") pod \"479424ad-3a30-4956-a828-94fe9c406ace\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") "
Sep 11 14:10:19.084444 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084371    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-model-cache\") pod \"479424ad-3a30-4956-a828-94fe9c406ace\" (UID: \"479424ad-3a30-4956-a828-94fe9c406ace\") "
Sep 11 14:10:19.084666 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084632    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "479424ad-3a30-4956-a828-94fe9c406ace" (UID: "479424ad-3a30-4956-a828-94fe9c406ace"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:19.084717 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084667    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-model-cache" (OuterVolumeSpecName: "model-cache") pod "479424ad-3a30-4956-a828-94fe9c406ace" (UID: "479424ad-3a30-4956-a828-94fe9c406ace"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:19.084717 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084637    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-home" (OuterVolumeSpecName: "home") pod "479424ad-3a30-4956-a828-94fe9c406ace" (UID: "479424ad-3a30-4956-a828-94fe9c406ace"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:19.084848 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084793    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:19.084848 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084808    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:19.084848 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.084816    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:19.086697 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.086674    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/479424ad-3a30-4956-a828-94fe9c406ace-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "479424ad-3a30-4956-a828-94fe9c406ace" (UID: "479424ad-3a30-4956-a828-94fe9c406ace"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:10:19.087013 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.086991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-dshm" (OuterVolumeSpecName: "dshm") pod "479424ad-3a30-4956-a828-94fe9c406ace" (UID: "479424ad-3a30-4956-a828-94fe9c406ace"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:19.087096 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.087033    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/479424ad-3a30-4956-a828-94fe9c406ace-kube-api-access-szz2j" (OuterVolumeSpecName: "kube-api-access-szz2j") pod "479424ad-3a30-4956-a828-94fe9c406ace" (UID: "479424ad-3a30-4956-a828-94fe9c406ace"). InnerVolumeSpecName "kube-api-access-szz2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:10:19.185515 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.185476    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/479424ad-3a30-4956-a828-94fe9c406ace-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:19.185515 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.185509    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/479424ad-3a30-4956-a828-94fe9c406ace-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:19.185515 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.185520    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szz2j\" (UniqueName: \"kubernetes.io/projected/479424ad-3a30-4956-a828-94fe9c406ace-kube-api-access-szz2j\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:19.772597 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.772543    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897" podUID="477a47eb-01d8-4c4c-abd7-16cf44f9508a" containerName="main" containerID="cri-o://54fc17f4a8564dd6be5c5f76fb61038190434a5db80cb83e10f6c696e38e932e" gracePeriod=45
Sep 11 14:10:19.965194 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.965164    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"
Sep 11 14:10:19.965384 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.965197    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k" event={"ID":"479424ad-3a30-4956-a828-94fe9c406ace","Type":"ContainerDied","Data":"032f1c4f714fc885b2aa98033c43389dbac043e4943cff2bc2a60149e44863e1"}
Sep 11 14:10:19.965384 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.965273    2563 scope.go:117] "RemoveContainer" containerID="374bf050e191bb37b96272d108b66fd71f54f9af54de8cdd8c50de13baca24a8"
Sep 11 14:10:19.967640 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.967603    2563 generic.go:356] "Generic (PLEG): container finished" podID="477a47eb-01d8-4c4c-abd7-16cf44f9508a" containerID="54fc17f4a8564dd6be5c5f76fb61038190434a5db80cb83e10f6c696e38e932e" exitCode=0
Sep 11 14:10:19.967786 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.967649    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897" event={"ID":"477a47eb-01d8-4c4c-abd7-16cf44f9508a","Type":"ContainerDied","Data":"54fc17f4a8564dd6be5c5f76fb61038190434a5db80cb83e10f6c696e38e932e"}
Sep 11 14:10:19.988975 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.988943    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"]
Sep 11 14:10:19.995287 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:19.995255    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/auth-test-model-932c97-kserve-dc544f6d7-xkr7k"]
Sep 11 14:10:20.038829 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.038742    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:10:20.195195 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195158    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tls-certs\") pod \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") "
Sep 11 14:10:20.195354 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195271    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-home\") pod \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") "
Sep 11 14:10:20.195354 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195316    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tmp-dir\") pod \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") "
Sep 11 14:10:20.195457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195356    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-dshm\") pod \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") "
Sep 11 14:10:20.195457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-model-cache\") pod \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") "
Sep 11 14:10:20.195457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195445    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n262z\" (UniqueName: \"kubernetes.io/projected/477a47eb-01d8-4c4c-abd7-16cf44f9508a-kube-api-access-n262z\") pod \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\" (UID: \"477a47eb-01d8-4c4c-abd7-16cf44f9508a\") "
Sep 11 14:10:20.195607 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195539    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "477a47eb-01d8-4c4c-abd7-16cf44f9508a" (UID: "477a47eb-01d8-4c4c-abd7-16cf44f9508a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:20.195607 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195585    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-home" (OuterVolumeSpecName: "home") pod "477a47eb-01d8-4c4c-abd7-16cf44f9508a" (UID: "477a47eb-01d8-4c4c-abd7-16cf44f9508a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:20.195730 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195714    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-model-cache" (OuterVolumeSpecName: "model-cache") pod "477a47eb-01d8-4c4c-abd7-16cf44f9508a" (UID: "477a47eb-01d8-4c4c-abd7-16cf44f9508a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:20.195848 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195829    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:20.195892 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195855    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:20.195892 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.195868    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:20.197812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.197780    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-dshm" (OuterVolumeSpecName: "dshm") pod "477a47eb-01d8-4c4c-abd7-16cf44f9508a" (UID: "477a47eb-01d8-4c4c-abd7-16cf44f9508a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:20.197812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.197799    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "477a47eb-01d8-4c4c-abd7-16cf44f9508a" (UID: "477a47eb-01d8-4c4c-abd7-16cf44f9508a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:10:20.197812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.197802    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477a47eb-01d8-4c4c-abd7-16cf44f9508a-kube-api-access-n262z" (OuterVolumeSpecName: "kube-api-access-n262z") pod "477a47eb-01d8-4c4c-abd7-16cf44f9508a" (UID: "477a47eb-01d8-4c4c-abd7-16cf44f9508a"). InnerVolumeSpecName "kube-api-access-n262z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:10:20.241168 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.241129    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="479424ad-3a30-4956-a828-94fe9c406ace" path="/var/lib/kubelet/pods/479424ad-3a30-4956-a828-94fe9c406ace/volumes"
Sep 11 14:10:20.297157 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.297054    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477a47eb-01d8-4c4c-abd7-16cf44f9508a-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:20.297157 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.297091    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n262z\" (UniqueName: \"kubernetes.io/projected/477a47eb-01d8-4c4c-abd7-16cf44f9508a-kube-api-access-n262z\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:20.297157 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.297106    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477a47eb-01d8-4c4c-abd7-16cf44f9508a-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:10:20.972650 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.972613    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"]
Sep 11 14:10:20.973085 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973055    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3ce7f67-7e7b-4065-9851-20ec08df28c8" containerName="revoke-keys"
Sep 11 14:10:20.973085 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973070    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3ce7f67-7e7b-4065-9851-20ec08df28c8" containerName="revoke-keys"
Sep 11 14:10:20.973085 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973082    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="477a47eb-01d8-4c4c-abd7-16cf44f9508a" containerName="main"
Sep 11 14:10:20.973085 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973088    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="477a47eb-01d8-4c4c-abd7-16cf44f9508a" containerName="main"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973098    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="202f8943-c9c4-41b6-8324-ae820fb1fef9" containerName="main"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973103    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="202f8943-c9c4-41b6-8324-ae820fb1fef9" containerName="main"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973114    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973119    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973132    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="479424ad-3a30-4956-a828-94fe9c406ace" containerName="main"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973137    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="479424ad-3a30-4956-a828-94fe9c406ace" containerName="main"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973151    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973157    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973210    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="479424ad-3a30-4956-a828-94fe9c406ace" containerName="main"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973219    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="477a47eb-01d8-4c4c-abd7-16cf44f9508a" containerName="main"
Sep 11 14:10:20.973224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973224    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy"
Sep 11 14:10:20.973558 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973230    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3ce7f67-7e7b-4065-9851-20ec08df28c8" containerName="revoke-keys"
Sep 11 14:10:20.973558 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.973237    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="202f8943-c9c4-41b6-8324-ae820fb1fef9" containerName="main"
Sep 11 14:10:20.975557 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.975534    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"
Sep 11 14:10:20.976505 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.976478    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897" event={"ID":"477a47eb-01d8-4c4c-abd7-16cf44f9508a","Type":"ContainerDied","Data":"339bacf0e9cdd8395c9d0dcfc7af0131073d6b33a9ccdf1028651e0bb036049a"}
Sep 11 14:10:20.976658 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.976524    2563 scope.go:117] "RemoveContainer" containerID="54fc17f4a8564dd6be5c5f76fb61038190434a5db80cb83e10f6c696e38e932e"
Sep 11 14:10:20.976658 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.976613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:20.979184 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.979157    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"openshift-service-ca.crt\""
Sep 11 14:10:20.980099 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.980081    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"kube-root-ca.crt\""
Sep 11 14:10:20.980229 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.980081    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"sub-test-model-932c97-kserve-self-signed-certs\""
Sep 11 14:10:20.980288 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.980237    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"default-dockercfg-zrsfg\""
Sep 11 14:10:20.995838 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:20.988776    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"]
Sep 11 14:10:21.008451 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.008415    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"]
Sep 11 14:10:21.011271 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.011240    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/auth-test-model-367517-kserve-7545c8c695-ss897"]
Sep 11 14:10:21.104488 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.104452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-home\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.104488 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.104493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-dshm\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.104731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.104528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpclx\" (UniqueName: \"kubernetes.io/projected/32238d46-196c-453e-aa00-696326d5d627-kube-api-access-vpclx\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.104731 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.104649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-tmp-dir\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.104847 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.104789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32238d46-196c-453e-aa00-696326d5d627-tls-certs\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.104897 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.104851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-model-cache\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.205957 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.205917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32238d46-196c-453e-aa00-696326d5d627-tls-certs\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.206172 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.205968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-model-cache\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.206172 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.206023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-home\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.206172 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.206046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-dshm\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.206172 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.206078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpclx\" (UniqueName: \"kubernetes.io/projected/32238d46-196c-453e-aa00-696326d5d627-kube-api-access-vpclx\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.206398 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.206243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-tmp-dir\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.206530 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.206510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-model-cache\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.206633 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.206611    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-home\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.206688 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.206625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-tmp-dir\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.208626 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.208604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-dshm\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.208733 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.208712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32238d46-196c-453e-aa00-696326d5d627-tls-certs\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.214210 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.214182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpclx\" (UniqueName: \"kubernetes.io/projected/32238d46-196c-453e-aa00-696326d5d627-kube-api-access-vpclx\") pod \"sub-test-model-932c97-kserve-559f749657-85kn7\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") " pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.293150 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.293048    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:21.569811 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.569782    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"]
Sep 11 14:10:21.571725 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:10:21.571695    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32238d46_196c_453e_aa00_696326d5d627.slice/crio-68db93fe5f6daa13c599efd42a1adecdd4367955fc61306932fd7e9c7b492e2d WatchSource:0}: Error finding container 68db93fe5f6daa13c599efd42a1adecdd4367955fc61306932fd7e9c7b492e2d: Status 404 returned error can't find the container with id 68db93fe5f6daa13c599efd42a1adecdd4367955fc61306932fd7e9c7b492e2d
Sep 11 14:10:21.984508 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.984472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7" event={"ID":"32238d46-196c-453e-aa00-696326d5d627","Type":"ContainerStarted","Data":"67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb"}
Sep 11 14:10:21.984508 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:21.984515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7" event={"ID":"32238d46-196c-453e-aa00-696326d5d627","Type":"ContainerStarted","Data":"68db93fe5f6daa13c599efd42a1adecdd4367955fc61306932fd7e9c7b492e2d"}
Sep 11 14:10:22.003162 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:22.003109    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7" podStartSLOduration=2.003092239 podStartE2EDuration="2.003092239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:10:22.002169874 +0000 UTC m=+1318.260369896" watchObservedRunningTime="2026-09-11 14:10:22.003092239 +0000 UTC m=+1318.261292247"
Sep 11 14:10:22.241252 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:22.241165    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="477a47eb-01d8-4c4c-abd7-16cf44f9508a" path="/var/lib/kubelet/pods/477a47eb-01d8-4c4c-abd7-16cf44f9508a/volumes"
Sep 11 14:10:24.433117 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:24.433086    2563 scope.go:117] "RemoveContainer" containerID="c3e2baae9b624a9bc3daed1a02baaf7edea2c0f0d36248ede946e6396a41acff"
Sep 11 14:10:24.442977 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:24.442948    2563 scope.go:117] "RemoveContainer" containerID="d1612ee5709797af03fdf6198f785bdf2bf52826e4ea572d2f5b1742253c83d8"
Sep 11 14:10:24.452050 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:24.452029    2563 scope.go:117] "RemoveContainer" containerID="acd4b051187b944a60535b0a5a1ee952b6b0f21cf3a0c271eeb49061bdf42ea0"
Sep 11 14:10:24.846456 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:24.846354    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:10:31.293626 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:31.293581    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:31.293626 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:31.293632    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:31.306588 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:31.306557    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:10:32.058907 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:10:32.058878    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:11:01.645140 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:01.645106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:11:02.456045 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.456011    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"]
Sep 11 14:11:02.456527 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.456512    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6806a5b-5d97-4e09-a052-ed7c37251727" containerName="istio-proxy"
Sep 11 14:11:02.460126 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.460102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.462958 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.462929    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"kube-root-ca.crt\""
Sep 11 14:11:02.464157 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.464137    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"openshift-service-ca.crt\""
Sep 11 14:11:02.464278 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.464155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"default-dockercfg-gwmlw\""
Sep 11 14:11:02.464278 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.464158    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"sub-test-model-367517-kserve-self-signed-certs\""
Sep 11 14:11:02.470578 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.470553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"]
Sep 11 14:11:02.610056 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.610016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-home\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.610056 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.610057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a86090-c891-456a-90ac-ab280b30dff3-tls-certs\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.610276 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.610144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-model-cache\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.610276 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.610221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-tmp-dir\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.610276 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.610250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkf2b\" (UniqueName: \"kubernetes.io/projected/60a86090-c891-456a-90ac-ab280b30dff3-kube-api-access-wkf2b\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.610276 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.610274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-dshm\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711339 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-tmp-dir\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711339 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkf2b\" (UniqueName: \"kubernetes.io/projected/60a86090-c891-456a-90ac-ab280b30dff3-kube-api-access-wkf2b\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-dshm\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-home\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a86090-c891-456a-90ac-ab280b30dff3-tls-certs\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-model-cache\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711626    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-tmp-dir\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711873    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-home\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.711891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.711883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-model-cache\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.713906 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.713880    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-dshm\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.714139 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.714123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a86090-c891-456a-90ac-ab280b30dff3-tls-certs\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.720906 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.720878    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkf2b\" (UniqueName: \"kubernetes.io/projected/60a86090-c891-456a-90ac-ab280b30dff3-kube-api-access-wkf2b\") pod \"sub-test-model-367517-kserve-844888cf7c-54pp6\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") " pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:02.771068 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:02.771018    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:03.049997 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:03.049969    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"]
Sep 11 14:11:03.051690 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:11:03.051662    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60a86090_c891_456a_90ac_ab280b30dff3.slice/crio-2b48a1381d32da21775b5c6b3092700422eaa15ee72c6566bc0e4f233e3c469c WatchSource:0}: Error finding container 2b48a1381d32da21775b5c6b3092700422eaa15ee72c6566bc0e4f233e3c469c: Status 404 returned error can't find the container with id 2b48a1381d32da21775b5c6b3092700422eaa15ee72c6566bc0e4f233e3c469c
Sep 11 14:11:03.242949 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:03.242914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6" event={"ID":"60a86090-c891-456a-90ac-ab280b30dff3","Type":"ContainerStarted","Data":"df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13"}
Sep 11 14:11:03.243156 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:03.242959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6" event={"ID":"60a86090-c891-456a-90ac-ab280b30dff3","Type":"ContainerStarted","Data":"2b48a1381d32da21775b5c6b3092700422eaa15ee72c6566bc0e4f233e3c469c"}
Sep 11 14:11:03.260575 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:03.260521    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6" podStartSLOduration=1.260499166 podStartE2EDuration="1.260499166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:03.260173911 +0000 UTC m=+1359.518373921" watchObservedRunningTime="2026-09-11 14:11:03.260499166 +0000 UTC m=+1359.518699174"
Sep 11 14:11:12.771847 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:12.771800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:12.771847 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:12.771856    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:12.784468 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:12.784431    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:13.314976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:13.314942    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:11:17.120062 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.120025    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"]
Sep 11 14:11:17.123827 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.123803    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.127081 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.127058    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1f2b49-openshift-default-dockercfg-fm4pm\""
Sep 11 14:11:17.140123 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.140087    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"]
Sep 11 14:11:17.257210 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bb1953c-03b3-4f4b-94de-0d18316738c7-istiod-ca-cert\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.257396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-token\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.257396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxg2m\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-kube-api-access-vxg2m\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.257396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-envoy\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.257396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-credential-socket\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.257396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-certs\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.257568 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-socket\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.257568 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-data\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.257568 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.257449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-podinfo\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358258 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-socket\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358258 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-data\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358533 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-podinfo\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358533 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bb1953c-03b3-4f4b-94de-0d18316738c7-istiod-ca-cert\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358533 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-token\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358533 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxg2m\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-kube-api-access-vxg2m\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358533 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-envoy\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358852 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-credential-socket\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358852 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-certs\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.358852 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-socket\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.359023 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-data\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.359023 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.358959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-credential-socket\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.359259 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.359237    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bb1953c-03b3-4f4b-94de-0d18316738c7-istiod-ca-cert\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.359384 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.359364    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-certs\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.361050 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.361023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-envoy\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.361322 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.361301    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-podinfo\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.367739 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.367709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-token\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.367887 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.367860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxg2m\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-kube-api-access-vxg2m\") pod \"e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") " pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.435869 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.435734    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:17.723848 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.723817    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"]
Sep 11 14:11:17.726720 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:11:17.726694    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb1953c_03b3_4f4b_94de_0d18316738c7.slice/crio-8376d77477e4f2f9684dbddbefa7a2bf3fef760bc14eb2796fe539f58744a350 WatchSource:0}: Error finding container 8376d77477e4f2f9684dbddbefa7a2bf3fef760bc14eb2796fe539f58744a350: Status 404 returned error can't find the container with id 8376d77477e4f2f9684dbddbefa7a2bf3fef760bc14eb2796fe539f58744a350
Sep 11 14:11:17.729257 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.729221    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:17.729345 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.729291    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:17.729388 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:17.729378    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:18.336601 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:18.336565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" event={"ID":"4bb1953c-03b3-4f4b-94de-0d18316738c7","Type":"ContainerStarted","Data":"aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb"}
Sep 11 14:11:18.337044 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:18.336617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" event={"ID":"4bb1953c-03b3-4f4b-94de-0d18316738c7","Type":"ContainerStarted","Data":"8376d77477e4f2f9684dbddbefa7a2bf3fef760bc14eb2796fe539f58744a350"}
Sep 11 14:11:18.359704 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:18.359642    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" podStartSLOduration=1.359623981 podStartE2EDuration="1.359623981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:18.357172142 +0000 UTC m=+1374.615372150" watchObservedRunningTime="2026-09-11 14:11:18.359623981 +0000 UTC m=+1374.617824032"
Sep 11 14:11:18.436833 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:18.436797    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:18.442444 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:18.442418    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:19.343310 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:19.343276    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:19.344329 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:19.344306    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:19.638951 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:19.638910    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:11:22.141652 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.141618    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"]
Sep 11 14:11:22.145522 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.145500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:22.148466 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.148391    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1f2b49\""
Sep 11 14:11:22.148697 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.148681    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-1f2b49-dockercfg-mwcxj\""
Sep 11 14:11:22.158031 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.158007    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"]
Sep 11 14:11:22.210591 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.210545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11668ef3-730c-48c5-bb8c-dc370f19c750-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg\" (UID: \"11668ef3-730c-48c5-bb8c-dc370f19c750\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:22.210782 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.210712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8k2s\" (UniqueName: \"kubernetes.io/projected/11668ef3-730c-48c5-bb8c-dc370f19c750-kube-api-access-p8k2s\") pod \"payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg\" (UID: \"11668ef3-730c-48c5-bb8c-dc370f19c750\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:22.312126 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.312084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8k2s\" (UniqueName: \"kubernetes.io/projected/11668ef3-730c-48c5-bb8c-dc370f19c750-kube-api-access-p8k2s\") pod \"payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg\" (UID: \"11668ef3-730c-48c5-bb8c-dc370f19c750\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:22.312319 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.312157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11668ef3-730c-48c5-bb8c-dc370f19c750-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg\" (UID: \"11668ef3-730c-48c5-bb8c-dc370f19c750\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:22.312958 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.312936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11668ef3-730c-48c5-bb8c-dc370f19c750-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg\" (UID: \"11668ef3-730c-48c5-bb8c-dc370f19c750\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:22.320799 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.320739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8k2s\" (UniqueName: \"kubernetes.io/projected/11668ef3-730c-48c5-bb8c-dc370f19c750-kube-api-access-p8k2s\") pod \"payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg\" (UID: \"11668ef3-730c-48c5-bb8c-dc370f19c750\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:22.457565 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.457465    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:22.737615 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:22.737517    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"]
Sep 11 14:11:22.740854 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:11:22.740824    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11668ef3_730c_48c5_bb8c_dc370f19c750.slice/crio-5cf0dfcae7c0fc3e0f0c37d3c449ace616a0fb04bae926862b90e42c1232cf56 WatchSource:0}: Error finding container 5cf0dfcae7c0fc3e0f0c37d3c449ace616a0fb04bae926862b90e42c1232cf56: Status 404 returned error can't find the container with id 5cf0dfcae7c0fc3e0f0c37d3c449ace616a0fb04bae926862b90e42c1232cf56
Sep 11 14:11:23.371591 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:23.371554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg" event={"ID":"11668ef3-730c-48c5-bb8c-dc370f19c750","Type":"ContainerStarted","Data":"5cf0dfcae7c0fc3e0f0c37d3c449ace616a0fb04bae926862b90e42c1232cf56"}
Sep 11 14:11:24.506870 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:24.505456    2563 scope.go:117] "RemoveContainer" containerID="8e1985cf0f2f9be100b352ce0c916bccf60fb6e7c5e6afa09e908d178cb638e1"
Sep 11 14:11:25.385327 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:25.385289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg" event={"ID":"11668ef3-730c-48c5-bb8c-dc370f19c750","Type":"ContainerStarted","Data":"ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce"}
Sep 11 14:11:25.385519 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:25.385471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:25.405088 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:25.405028    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg" podStartSLOduration=1.651002691 podStartE2EDuration="3.405005436s" totalImagesPullingTime="1.754002745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:25.401937558 +0000 UTC m=+1381.660137566" watchObservedRunningTime="2026-09-11 14:11:25.405005436 +0000 UTC m=+1381.663205443"
Sep 11 14:11:36.394607 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:36.394569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:11:38.476837 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:38.476784    2563 generic.go:356] "Generic (PLEG): container finished" podID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerID="aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb" exitCode=0
Sep 11 14:11:38.477279 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:38.476865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" event={"ID":"4bb1953c-03b3-4f4b-94de-0d18316738c7","Type":"ContainerDied","Data":"aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb"}
Sep 11 14:11:38.477398 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:38.477382    2563 scope.go:117] "RemoveContainer" containerID="aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb"
Sep 11 14:11:38.478578 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:38.478551    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:38.478636 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:38.478626    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:38.478672 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:38.478660    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:39.147544 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:39.147506    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:11:39.485184 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:39.485096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" event={"ID":"4bb1953c-03b3-4f4b-94de-0d18316738c7","Type":"ContainerStarted","Data":"5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b"}
Sep 11 14:11:40.436427 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:40.436385    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:40.436635 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:40.436616    2563 patch_prober.go:28] interesting pod/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 14:11:40.436680 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:40.436665    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 14:11:41.436487 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:41.436443    2563 patch_prober.go:28] interesting pod/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 14:11:41.436971 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:41.436502    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 14:11:42.436485 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.436445    2563 patch_prober.go:28] interesting pod/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 14:11:42.436678 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.436509    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 14:11:42.477286 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.477244    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7543-4a9d1d"]
Sep 11 14:11:42.480144 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.480105    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-4a9d1d"
Sep 11 14:11:42.484053 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.484026    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2qcb7\""
Sep 11 14:11:42.489291 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.489263    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-4a9d1d"]
Sep 11 14:11:42.533466 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.533410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-426t7\" (UniqueName: \"kubernetes.io/projected/6bb94de3-1f17-495d-8760-3180637a1da0-kube-api-access-426t7\") pod \"test-curl-7543-4a9d1d\" (UID: \"6bb94de3-1f17-495d-8760-3180637a1da0\") " pod="opendatahub/test-curl-7543-4a9d1d"
Sep 11 14:11:42.634365 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.634327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-426t7\" (UniqueName: \"kubernetes.io/projected/6bb94de3-1f17-495d-8760-3180637a1da0-kube-api-access-426t7\") pod \"test-curl-7543-4a9d1d\" (UID: \"6bb94de3-1f17-495d-8760-3180637a1da0\") " pod="opendatahub/test-curl-7543-4a9d1d"
Sep 11 14:11:42.642890 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.642864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-426t7\" (UniqueName: \"kubernetes.io/projected/6bb94de3-1f17-495d-8760-3180637a1da0-kube-api-access-426t7\") pod \"test-curl-7543-4a9d1d\" (UID: \"6bb94de3-1f17-495d-8760-3180637a1da0\") " pod="opendatahub/test-curl-7543-4a9d1d"
Sep 11 14:11:42.795212 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:42.795104    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-4a9d1d"
Sep 11 14:11:43.088991 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:43.088966    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-4a9d1d"]
Sep 11 14:11:43.090967 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:11:43.090926    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bb94de3_1f17_495d_8760_3180637a1da0.slice/crio-0fff91f3936a3b3e2467bfd657ae2ab3cb567df2f26333c089b882bbf0b4e2e8 WatchSource:0}: Error finding container 0fff91f3936a3b3e2467bfd657ae2ab3cb567df2f26333c089b882bbf0b4e2e8: Status 404 returned error can't find the container with id 0fff91f3936a3b3e2467bfd657ae2ab3cb567df2f26333c089b882bbf0b4e2e8
Sep 11 14:11:43.436480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:43.436443    2563 patch_prober.go:28] interesting pod/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 14:11:43.436676 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:43.436500    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 14:11:43.524312 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:43.524260    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-4a9d1d" event={"ID":"6bb94de3-1f17-495d-8760-3180637a1da0","Type":"ContainerStarted","Data":"50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201"}
Sep 11 14:11:43.524860 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:43.524326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-4a9d1d" event={"ID":"6bb94de3-1f17-495d-8760-3180637a1da0","Type":"ContainerStarted","Data":"0fff91f3936a3b3e2467bfd657ae2ab3cb567df2f26333c089b882bbf0b4e2e8"}
Sep 11 14:11:43.540767 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:43.540709    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7543-4a9d1d" podStartSLOduration=1.540693429 podStartE2EDuration="1.540693429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:43.540043488 +0000 UTC m=+1399.798243495" watchObservedRunningTime="2026-09-11 14:11:43.540693429 +0000 UTC m=+1399.798893436"
Sep 11 14:11:44.174598 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.174557    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7543-4a9d1d"]
Sep 11 14:11:44.179043 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.179013    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7543-4a9d1d"]
Sep 11 14:11:44.299552 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.299466    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7543-18a547"]
Sep 11 14:11:44.299970 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.299956    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bb94de3-1f17-495d-8760-3180637a1da0" containerName="test-curl-7543-4a9d1d"
Sep 11 14:11:44.300021 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.299972    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bb94de3-1f17-495d-8760-3180637a1da0" containerName="test-curl-7543-4a9d1d"
Sep 11 14:11:44.300057 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.300041    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bb94de3-1f17-495d-8760-3180637a1da0" containerName="test-curl-7543-4a9d1d"
Sep 11 14:11:44.316205 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.316161    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-18a547"
Sep 11 14:11:44.335512 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.334964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-18a547"]
Sep 11 14:11:44.350503 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.350472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6552\" (UniqueName: \"kubernetes.io/projected/a05e6cee-f7f6-450b-8e52-237d7df16963-kube-api-access-c6552\") pod \"test-curl-7543-18a547\" (UID: \"a05e6cee-f7f6-450b-8e52-237d7df16963\") " pod="opendatahub/test-curl-7543-18a547"
Sep 11 14:11:44.436657 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.436618    2563 patch_prober.go:28] interesting pod/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 14:11:44.436904 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.436675    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 14:11:44.451196 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.451159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6552\" (UniqueName: \"kubernetes.io/projected/a05e6cee-f7f6-450b-8e52-237d7df16963-kube-api-access-c6552\") pod \"test-curl-7543-18a547\" (UID: \"a05e6cee-f7f6-450b-8e52-237d7df16963\") " pod="opendatahub/test-curl-7543-18a547"
Sep 11 14:11:44.460169 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.460102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6552\" (UniqueName: \"kubernetes.io/projected/a05e6cee-f7f6-450b-8e52-237d7df16963-kube-api-access-c6552\") pod \"test-curl-7543-18a547\" (UID: \"a05e6cee-f7f6-450b-8e52-237d7df16963\") " pod="opendatahub/test-curl-7543-18a547"
Sep 11 14:11:44.532188 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:11:44.532147    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7543-4a9d1d" container="test-curl-7543-4a9d1d"
Sep 11 14:11:44.639768 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.639712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-18a547"
Sep 11 14:11:44.939605 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:44.939513    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-18a547"]
Sep 11 14:11:44.942890 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:11:44.942848    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda05e6cee_f7f6_450b_8e52_237d7df16963.slice/crio-3b8e02c1450bf484bc485dbaa16e45259052a7bc0ffa525998913f2ad9f0102b WatchSource:0}: Error finding container 3b8e02c1450bf484bc485dbaa16e45259052a7bc0ffa525998913f2ad9f0102b: Status 404 returned error can't find the container with id 3b8e02c1450bf484bc485dbaa16e45259052a7bc0ffa525998913f2ad9f0102b
Sep 11 14:11:45.440531 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:45.440499    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:45.440815 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:45.440794    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:45.441609 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:45.441593    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:11:45.541579 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:45.541543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-18a547" event={"ID":"a05e6cee-f7f6-450b-8e52-237d7df16963","Type":"ContainerStarted","Data":"576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7"}
Sep 11 14:11:45.542083 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:45.541589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-18a547" event={"ID":"a05e6cee-f7f6-450b-8e52-237d7df16963","Type":"ContainerStarted","Data":"3b8e02c1450bf484bc485dbaa16e45259052a7bc0ffa525998913f2ad9f0102b"}
Sep 11 14:11:45.542083 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:45.541883    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7543-4a9d1d" podUID="6bb94de3-1f17-495d-8760-3180637a1da0" containerName="test-curl-7543-4a9d1d" containerID="cri-o://50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201" gracePeriod=2
Sep 11 14:11:45.557117 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:45.557068    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7543-18a547" podStartSLOduration=1.557050244 podStartE2EDuration="1.557050244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:45.554069301 +0000 UTC m=+1401.812269315" watchObservedRunningTime="2026-09-11 14:11:45.557050244 +0000 UTC m=+1401.815250633"
Sep 11 14:11:46.135287 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:46.135250    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7543-18a547"]
Sep 11 14:11:46.140202 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:46.140173    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7543-18a547"]
Sep 11 14:11:46.548068 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:11:46.547970    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7543-18a547" container="test-curl-7543-18a547"
Sep 11 14:11:47.553682 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:47.553644    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7543-18a547" podUID="a05e6cee-f7f6-450b-8e52-237d7df16963" containerName="test-curl-7543-18a547" containerID="cri-o://576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7" gracePeriod=2
Sep 11 14:11:47.695665 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:47.695640    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-4a9d1d"
Sep 11 14:11:47.786048 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:47.786018    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-426t7\" (UniqueName: \"kubernetes.io/projected/6bb94de3-1f17-495d-8760-3180637a1da0-kube-api-access-426t7\") pod \"6bb94de3-1f17-495d-8760-3180637a1da0\" (UID: \"6bb94de3-1f17-495d-8760-3180637a1da0\") "
Sep 11 14:11:47.788346 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:47.788317    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bb94de3-1f17-495d-8760-3180637a1da0-kube-api-access-426t7" (OuterVolumeSpecName: "kube-api-access-426t7") pod "6bb94de3-1f17-495d-8760-3180637a1da0" (UID: "6bb94de3-1f17-495d-8760-3180637a1da0"). InnerVolumeSpecName "kube-api-access-426t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:11:47.887595 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:47.887554    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-426t7\" (UniqueName: \"kubernetes.io/projected/6bb94de3-1f17-495d-8760-3180637a1da0-kube-api-access-426t7\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:11:48.045079 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.045045    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"]
Sep 11 14:11:48.045493 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.045480    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a05e6cee-f7f6-450b-8e52-237d7df16963" containerName="test-curl-7543-18a547"
Sep 11 14:11:48.045539 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.045495    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a05e6cee-f7f6-450b-8e52-237d7df16963" containerName="test-curl-7543-18a547"
Sep 11 14:11:48.045584 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.045572    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a05e6cee-f7f6-450b-8e52-237d7df16963" containerName="test-curl-7543-18a547"
Sep 11 14:11:48.047667 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.047645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"
Sep 11 14:11:48.050344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.050322    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:11:48.050500 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.050324    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:11:48.056886 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.056864    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"]
Sep 11 14:11:48.089506 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.089476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3b5355-bf24-4f16-b51f-658cb5bfd8c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-427751-fpgjg\" (UID: \"ff3b5355-bf24-4f16-b51f-658cb5bfd8c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"
Sep 11 14:11:48.124167 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.124135    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"]
Sep 11 14:11:48.190781 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.190672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3b5355-bf24-4f16-b51f-658cb5bfd8c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-427751-fpgjg\" (UID: \"ff3b5355-bf24-4f16-b51f-658cb5bfd8c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"
Sep 11 14:11:48.191401 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.191378    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3b5355-bf24-4f16-b51f-658cb5bfd8c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-427751-fpgjg\" (UID: \"ff3b5355-bf24-4f16-b51f-658cb5bfd8c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"
Sep 11 14:11:48.256332 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.256301    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bb94de3-1f17-495d-8760-3180637a1da0" path="/var/lib/kubelet/pods/6bb94de3-1f17-495d-8760-3180637a1da0/volumes"
Sep 11 14:11:48.360912 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.360881    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"
Sep 11 14:11:48.561900 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.561805    2563 generic.go:356] "Generic (PLEG): container finished" podID="6bb94de3-1f17-495d-8760-3180637a1da0" containerID="50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201" exitCode=137
Sep 11 14:11:48.561900 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.561856    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-4a9d1d"
Sep 11 14:11:48.561900 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.561894    2563 scope.go:117] "RemoveContainer" containerID="50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201"
Sep 11 14:11:48.571186 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.571165    2563 scope.go:117] "RemoveContainer" containerID="50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201"
Sep 11 14:11:48.571518 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:11:48.571499    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201\": container with ID starting with 50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201 not found: ID does not exist" containerID="50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201"
Sep 11 14:11:48.571580 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.571527    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201"} err="failed to get container status \"50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201\": rpc error: code = NotFound desc = could not find container \"50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201\": container with ID starting with 50f5016eefaa21cfed796ab21dac9d39c83c5544b0eb6b9901fac710d07df201 not found: ID does not exist"
Sep 11 14:11:48.634679 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:48.634652    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"]
Sep 11 14:11:48.636333 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:11:48.636299    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3b5355_bf24_4f16_b51f_658cb5bfd8c7.slice/crio-3f9c6b7b23a2ea0fc3548e1581f2b19a365ad823c242e716d27dbdc5ceb3ba53 WatchSource:0}: Error finding container 3f9c6b7b23a2ea0fc3548e1581f2b19a365ad823c242e716d27dbdc5ceb3ba53: Status 404 returned error can't find the container with id 3f9c6b7b23a2ea0fc3548e1581f2b19a365ad823c242e716d27dbdc5ceb3ba53
Sep 11 14:11:49.167472 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:49.167437    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"]
Sep 11 14:11:49.571845 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:49.571814    2563 generic.go:356] "Generic (PLEG): container finished" podID="ff3b5355-bf24-4f16-b51f-658cb5bfd8c7" containerID="6104e3962f6c9cc0fa9e8b6b237ff1462b3be9810bed5b77eecdb03260ca390b" exitCode=0
Sep 11 14:11:49.572209 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:49.571874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg" event={"ID":"ff3b5355-bf24-4f16-b51f-658cb5bfd8c7","Type":"ContainerDied","Data":"6104e3962f6c9cc0fa9e8b6b237ff1462b3be9810bed5b77eecdb03260ca390b"}
Sep 11 14:11:49.572209 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:49.571926    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg" event={"ID":"ff3b5355-bf24-4f16-b51f-658cb5bfd8c7","Type":"ContainerStarted","Data":"3f9c6b7b23a2ea0fc3548e1581f2b19a365ad823c242e716d27dbdc5ceb3ba53"}
Sep 11 14:11:49.749379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:49.749355    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-18a547"
Sep 11 14:11:49.816168 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:49.816077    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6552\" (UniqueName: \"kubernetes.io/projected/a05e6cee-f7f6-450b-8e52-237d7df16963-kube-api-access-c6552\") pod \"a05e6cee-f7f6-450b-8e52-237d7df16963\" (UID: \"a05e6cee-f7f6-450b-8e52-237d7df16963\") "
Sep 11 14:11:49.818362 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:49.818329    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a05e6cee-f7f6-450b-8e52-237d7df16963-kube-api-access-c6552" (OuterVolumeSpecName: "kube-api-access-c6552") pod "a05e6cee-f7f6-450b-8e52-237d7df16963" (UID: "a05e6cee-f7f6-450b-8e52-237d7df16963"). InnerVolumeSpecName "kube-api-access-c6552". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:11:49.917038 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:49.916999    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6552\" (UniqueName: \"kubernetes.io/projected/a05e6cee-f7f6-450b-8e52-237d7df16963-kube-api-access-c6552\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:11:50.241870 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.241839    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a05e6cee-f7f6-450b-8e52-237d7df16963" path="/var/lib/kubelet/pods/a05e6cee-f7f6-450b-8e52-237d7df16963/volumes"
Sep 11 14:11:50.579264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.579178    2563 generic.go:356] "Generic (PLEG): container finished" podID="a05e6cee-f7f6-450b-8e52-237d7df16963" containerID="576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7" exitCode=137
Sep 11 14:11:50.579264 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.579233    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-18a547"
Sep 11 14:11:50.579804 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.579282    2563 scope.go:117] "RemoveContainer" containerID="576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7"
Sep 11 14:11:50.589190 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.589162    2563 scope.go:117] "RemoveContainer" containerID="576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7"
Sep 11 14:11:50.589581 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:11:50.589543    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7\": container with ID starting with 576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7 not found: ID does not exist" containerID="576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7"
Sep 11 14:11:50.589716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.589594    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7"} err="failed to get container status \"576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7\": rpc error: code = NotFound desc = could not find container \"576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7\": container with ID starting with 576b068299682029aecf7bca1b676bf3240631e6672eb8f93025081d2b1b91a7 not found: ID does not exist"
Sep 11 14:11:50.729142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.729114    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"
Sep 11 14:11:50.826420 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.826388    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3b5355-bf24-4f16-b51f-658cb5bfd8c7-maas-api-service-ca\") pod \"ff3b5355-bf24-4f16-b51f-658cb5bfd8c7\" (UID: \"ff3b5355-bf24-4f16-b51f-658cb5bfd8c7\") "
Sep 11 14:11:50.826867 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.826842    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3b5355-bf24-4f16-b51f-658cb5bfd8c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff3b5355-bf24-4f16-b51f-658cb5bfd8c7" (UID: "ff3b5355-bf24-4f16-b51f-658cb5bfd8c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:11:50.928084 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:50.928040    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3b5355-bf24-4f16-b51f-658cb5bfd8c7-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:11:51.587787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:51.587722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg" event={"ID":"ff3b5355-bf24-4f16-b51f-658cb5bfd8c7","Type":"ContainerDied","Data":"3f9c6b7b23a2ea0fc3548e1581f2b19a365ad823c242e716d27dbdc5ceb3ba53"}
Sep 11 14:11:51.588290 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:51.587805    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f9c6b7b23a2ea0fc3548e1581f2b19a365ad823c242e716d27dbdc5ceb3ba53"
Sep 11 14:11:51.588290 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:51.587736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"
Sep 11 14:11:53.048030 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.047997    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"]
Sep 11 14:11:53.048525 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.048433    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff3b5355-bf24-4f16-b51f-658cb5bfd8c7" containerName="revoke-keys"
Sep 11 14:11:53.048525 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.048447    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff3b5355-bf24-4f16-b51f-658cb5bfd8c7" containerName="revoke-keys"
Sep 11 14:11:53.048654 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.048543    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff3b5355-bf24-4f16-b51f-658cb5bfd8c7" containerName="revoke-keys"
Sep 11 14:11:53.050711 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.050687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.053192 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.053171    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-932c97\"/\"rate-test-model-932c97-kserve-self-signed-certs\""
Sep 11 14:11:53.063039 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.063012    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"]
Sep 11 14:11:53.138525 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.138489    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:11:53.148344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.148318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlb7d\" (UniqueName: \"kubernetes.io/projected/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-kube-api-access-qlb7d\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.148467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.148361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-dshm\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.148467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.148405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-model-cache\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.148467 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.148462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-home\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.148577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.148493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tls-certs\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.148577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.148521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tmp-dir\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249145 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tmp-dir\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249329 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlb7d\" (UniqueName: \"kubernetes.io/projected/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-kube-api-access-qlb7d\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249329 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-dshm\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249329 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-model-cache\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249478 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-home\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249478 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tls-certs\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249589 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249526    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tmp-dir\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249774 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-model-cache\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.249889 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.249829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-home\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.251664 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.251643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-dshm\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.252438 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.252421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tls-certs\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.258224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.258199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlb7d\" (UniqueName: \"kubernetes.io/projected/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-kube-api-access-qlb7d\") pod \"rate-test-model-932c97-kserve-6d59f56d99-5bjv4\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") " pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.360522 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.360471    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:11:53.644320 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:53.644182    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"]
Sep 11 14:11:53.650138 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:11:53.650105    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2550d4ba_6e9a_49a9_bc17_959b42ebb4d8.slice/crio-4e9dd7fb3952e01d1ba98ea4dae9ae41995e060e02695822412bef60e153da49 WatchSource:0}: Error finding container 4e9dd7fb3952e01d1ba98ea4dae9ae41995e060e02695822412bef60e153da49: Status 404 returned error can't find the container with id 4e9dd7fb3952e01d1ba98ea4dae9ae41995e060e02695822412bef60e153da49
Sep 11 14:11:54.610117 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:54.610067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4" event={"ID":"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8","Type":"ContainerStarted","Data":"26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548"}
Sep 11 14:11:54.610528 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:54.610124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4" event={"ID":"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8","Type":"ContainerStarted","Data":"4e9dd7fb3952e01d1ba98ea4dae9ae41995e060e02695822412bef60e153da49"}
Sep 11 14:11:54.630267 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:11:54.630218    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4" podStartSLOduration=1.630200167 podStartE2EDuration="1.630200167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:54.628665629 +0000 UTC m=+1410.886865640" watchObservedRunningTime="2026-09-11 14:11:54.630200167 +0000 UTC m=+1410.888400175"
Sep 11 14:12:03.145022 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.144955    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7" podUID="32238d46-196c-453e-aa00-696326d5d627" containerName="main" containerID="cri-o://67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb" gracePeriod=45
Sep 11 14:12:03.361083 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.361050    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:12:03.361083 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.361094    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:12:03.376111 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.376081    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:12:03.420778 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.420722    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:12:03.555860 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.555816    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpclx\" (UniqueName: \"kubernetes.io/projected/32238d46-196c-453e-aa00-696326d5d627-kube-api-access-vpclx\") pod \"32238d46-196c-453e-aa00-696326d5d627\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") "
Sep 11 14:12:03.555860 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.555867    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32238d46-196c-453e-aa00-696326d5d627-tls-certs\") pod \"32238d46-196c-453e-aa00-696326d5d627\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") "
Sep 11 14:12:03.556133 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.555989    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-tmp-dir\") pod \"32238d46-196c-453e-aa00-696326d5d627\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") "
Sep 11 14:12:03.556133 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556037    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-home\") pod \"32238d46-196c-453e-aa00-696326d5d627\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") "
Sep 11 14:12:03.556133 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556069    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-dshm\") pod \"32238d46-196c-453e-aa00-696326d5d627\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") "
Sep 11 14:12:03.556133 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556092    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-model-cache\") pod \"32238d46-196c-453e-aa00-696326d5d627\" (UID: \"32238d46-196c-453e-aa00-696326d5d627\") "
Sep 11 14:12:03.556331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556295    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-home" (OuterVolumeSpecName: "home") pod "32238d46-196c-453e-aa00-696326d5d627" (UID: "32238d46-196c-453e-aa00-696326d5d627"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:03.556331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556306    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "32238d46-196c-453e-aa00-696326d5d627" (UID: "32238d46-196c-453e-aa00-696326d5d627"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:03.556452 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556352    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-model-cache" (OuterVolumeSpecName: "model-cache") pod "32238d46-196c-453e-aa00-696326d5d627" (UID: "32238d46-196c-453e-aa00-696326d5d627"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:03.556510 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556497    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:03.556571 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556512    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:03.556571 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.556527    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:03.558316 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.558286    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-dshm" (OuterVolumeSpecName: "dshm") pod "32238d46-196c-453e-aa00-696326d5d627" (UID: "32238d46-196c-453e-aa00-696326d5d627"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:03.558436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.558335    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32238d46-196c-453e-aa00-696326d5d627-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "32238d46-196c-453e-aa00-696326d5d627" (UID: "32238d46-196c-453e-aa00-696326d5d627"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:12:03.558436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.558342    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32238d46-196c-453e-aa00-696326d5d627-kube-api-access-vpclx" (OuterVolumeSpecName: "kube-api-access-vpclx") pod "32238d46-196c-453e-aa00-696326d5d627" (UID: "32238d46-196c-453e-aa00-696326d5d627"). InnerVolumeSpecName "kube-api-access-vpclx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:03.657700 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.657588    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32238d46-196c-453e-aa00-696326d5d627-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:03.657700 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.657635    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpclx\" (UniqueName: \"kubernetes.io/projected/32238d46-196c-453e-aa00-696326d5d627-kube-api-access-vpclx\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:03.657700 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.657646    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32238d46-196c-453e-aa00-696326d5d627-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:03.668187 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.668156    2563 generic.go:356] "Generic (PLEG): container finished" podID="32238d46-196c-453e-aa00-696326d5d627" containerID="67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb" exitCode=0
Sep 11 14:12:03.668356 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.668203    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"
Sep 11 14:12:03.668356 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.668233    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7" event={"ID":"32238d46-196c-453e-aa00-696326d5d627","Type":"ContainerDied","Data":"67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb"}
Sep 11 14:12:03.668356 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.668278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7" event={"ID":"32238d46-196c-453e-aa00-696326d5d627","Type":"ContainerDied","Data":"68db93fe5f6daa13c599efd42a1adecdd4367955fc61306932fd7e9c7b492e2d"}
Sep 11 14:12:03.668356 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.668298    2563 scope.go:117] "RemoveContainer" containerID="67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb"
Sep 11 14:12:03.682807 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.682783    2563 scope.go:117] "RemoveContainer" containerID="67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb"
Sep 11 14:12:03.683438 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:12:03.683411    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb\": container with ID starting with 67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb not found: ID does not exist" containerID="67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb"
Sep 11 14:12:03.683543 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.683446    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb"} err="failed to get container status \"67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb\": rpc error: code = NotFound desc = could not find container \"67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb\": container with ID starting with 67923e8ef26bc6c68a5976dcc74cd4e0c314af03c2e6f36e4bac042ed4da25bb not found: ID does not exist"
Sep 11 14:12:03.683772 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.683728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:12:03.696066 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.696035    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"]
Sep 11 14:12:03.699266 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:03.699240    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/sub-test-model-932c97-kserve-559f749657-85kn7"]
Sep 11 14:12:04.049910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.049819    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"]
Sep 11 14:12:04.052358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.052337    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-427751-fpgjg"]
Sep 11 14:12:04.206152 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.206108    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6" podUID="60a86090-c891-456a-90ac-ab280b30dff3" containerName="main" containerID="cri-o://df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13" gracePeriod=45
Sep 11 14:12:04.242157 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.242127    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32238d46-196c-453e-aa00-696326d5d627" path="/var/lib/kubelet/pods/32238d46-196c-453e-aa00-696326d5d627/volumes"
Sep 11 14:12:04.242487 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.242475    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff3b5355-bf24-4f16-b51f-658cb5bfd8c7" path="/var/lib/kubelet/pods/ff3b5355-bf24-4f16-b51f-658cb5bfd8c7/volumes"
Sep 11 14:12:04.472513 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.472488    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:12:04.567833 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.567789    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-home\") pod \"60a86090-c891-456a-90ac-ab280b30dff3\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") "
Sep 11 14:12:04.568058 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.567851    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-model-cache\") pod \"60a86090-c891-456a-90ac-ab280b30dff3\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") "
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.568336    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-dshm\") pod \"60a86090-c891-456a-90ac-ab280b30dff3\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") "
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.568404    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-model-cache" (OuterVolumeSpecName: "model-cache") pod "60a86090-c891-456a-90ac-ab280b30dff3" (UID: "60a86090-c891-456a-90ac-ab280b30dff3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.568417    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-home" (OuterVolumeSpecName: "home") pod "60a86090-c891-456a-90ac-ab280b30dff3" (UID: "60a86090-c891-456a-90ac-ab280b30dff3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.568587    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkf2b\" (UniqueName: \"kubernetes.io/projected/60a86090-c891-456a-90ac-ab280b30dff3-kube-api-access-wkf2b\") pod \"60a86090-c891-456a-90ac-ab280b30dff3\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") "
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.568635    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-tmp-dir\") pod \"60a86090-c891-456a-90ac-ab280b30dff3\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") "
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.568734    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a86090-c891-456a-90ac-ab280b30dff3-tls-certs\") pod \"60a86090-c891-456a-90ac-ab280b30dff3\" (UID: \"60a86090-c891-456a-90ac-ab280b30dff3\") "
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.569339    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.569367    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.571181    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-dshm" (OuterVolumeSpecName: "dshm") pod "60a86090-c891-456a-90ac-ab280b30dff3" (UID: "60a86090-c891-456a-90ac-ab280b30dff3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.571911    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60a86090-c891-456a-90ac-ab280b30dff3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60a86090-c891-456a-90ac-ab280b30dff3" (UID: "60a86090-c891-456a-90ac-ab280b30dff3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:12:04.572992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.572127    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60a86090-c891-456a-90ac-ab280b30dff3" (UID: "60a86090-c891-456a-90ac-ab280b30dff3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:04.575823 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.573717    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a86090-c891-456a-90ac-ab280b30dff3-kube-api-access-wkf2b" (OuterVolumeSpecName: "kube-api-access-wkf2b") pod "60a86090-c891-456a-90ac-ab280b30dff3" (UID: "60a86090-c891-456a-90ac-ab280b30dff3"). InnerVolumeSpecName "kube-api-access-wkf2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:04.669988 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.669954    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.669988 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.669986    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkf2b\" (UniqueName: \"kubernetes.io/projected/60a86090-c891-456a-90ac-ab280b30dff3-kube-api-access-wkf2b\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.670217 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.670003    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60a86090-c891-456a-90ac-ab280b30dff3-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.670217 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.670018    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a86090-c891-456a-90ac-ab280b30dff3-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.675618 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.675580    2563 generic.go:356] "Generic (PLEG): container finished" podID="60a86090-c891-456a-90ac-ab280b30dff3" containerID="df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13" exitCode=0
Sep 11 14:12:04.675812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.675642    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"
Sep 11 14:12:04.675812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.675675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6" event={"ID":"60a86090-c891-456a-90ac-ab280b30dff3","Type":"ContainerDied","Data":"df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13"}
Sep 11 14:12:04.675812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.675723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6" event={"ID":"60a86090-c891-456a-90ac-ab280b30dff3","Type":"ContainerDied","Data":"2b48a1381d32da21775b5c6b3092700422eaa15ee72c6566bc0e4f233e3c469c"}
Sep 11 14:12:04.675812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.675745    2563 scope.go:117] "RemoveContainer" containerID="df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13"
Sep 11 14:12:04.687337 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.687311    2563 scope.go:117] "RemoveContainer" containerID="df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13"
Sep 11 14:12:04.687575 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:12:04.687556    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13\": container with ID starting with df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13 not found: ID does not exist" containerID="df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13"
Sep 11 14:12:04.687643 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.687582    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13"} err="failed to get container status \"df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13\": rpc error: code = NotFound desc = could not find container \"df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13\": container with ID starting with df5d15ce7aa44a0506c8f15775212d5033b9029448bab7e426757e1134fedc13 not found: ID does not exist"
Sep 11 14:12:04.700856 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.700829    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"]
Sep 11 14:12:04.718141 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:04.718113    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/sub-test-model-367517-kserve-844888cf7c-54pp6"]
Sep 11 14:12:06.241597 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:06.241560    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60a86090-c891-456a-90ac-ab280b30dff3" path="/var/lib/kubelet/pods/60a86090-c891-456a-90ac-ab280b30dff3/volumes"
Sep 11 14:12:09.351453 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:09.351413    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:12:11.146894 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.146854    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"]
Sep 11 14:12:11.147288 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.147243    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32238d46-196c-453e-aa00-696326d5d627" containerName="main"
Sep 11 14:12:11.147288 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.147253    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="32238d46-196c-453e-aa00-696326d5d627" containerName="main"
Sep 11 14:12:11.147288 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.147274    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a86090-c891-456a-90ac-ab280b30dff3" containerName="main"
Sep 11 14:12:11.147288 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.147279    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a86090-c891-456a-90ac-ab280b30dff3" containerName="main"
Sep 11 14:12:11.147422 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.147354    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a86090-c891-456a-90ac-ab280b30dff3" containerName="main"
Sep 11 14:12:11.147422 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.147362    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="32238d46-196c-453e-aa00-696326d5d627" containerName="main"
Sep 11 14:12:11.150270 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.150251    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"
Sep 11 14:12:11.153008 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.152988    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:12:11.153151 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.152988    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:12:11.158876 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.158851    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"]
Sep 11 14:12:11.341067 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.341023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a887c26-971a-4e90-b7b5-2ca6e1764f99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn\" (UID: \"1a887c26-971a-4e90-b7b5-2ca6e1764f99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"
Sep 11 14:12:11.442389 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.442297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a887c26-971a-4e90-b7b5-2ca6e1764f99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn\" (UID: \"1a887c26-971a-4e90-b7b5-2ca6e1764f99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"
Sep 11 14:12:11.443069 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.443045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a887c26-971a-4e90-b7b5-2ca6e1764f99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn\" (UID: \"1a887c26-971a-4e90-b7b5-2ca6e1764f99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"
Sep 11 14:12:11.462402 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.462369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"
Sep 11 14:12:11.742551 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:11.742523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"]
Sep 11 14:12:11.744390 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:12:11.744349    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a887c26_971a_4e90_b7b5_2ca6e1764f99.slice/crio-482e4102ad90c073aba16c1ebba3f2d7584585c5743b2e8a588906f430ffb8a6 WatchSource:0}: Error finding container 482e4102ad90c073aba16c1ebba3f2d7584585c5743b2e8a588906f430ffb8a6: Status 404 returned error can't find the container with id 482e4102ad90c073aba16c1ebba3f2d7584585c5743b2e8a588906f430ffb8a6
Sep 11 14:12:12.739521 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:12.739427    2563 generic.go:356] "Generic (PLEG): container finished" podID="1a887c26-971a-4e90-b7b5-2ca6e1764f99" containerID="6883cb501da6e277f76f700ef772c046f368c01070cdb2948e83d1ce74944986" exitCode=0
Sep 11 14:12:12.739521 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:12.739503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn" event={"ID":"1a887c26-971a-4e90-b7b5-2ca6e1764f99","Type":"ContainerDied","Data":"6883cb501da6e277f76f700ef772c046f368c01070cdb2948e83d1ce74944986"}
Sep 11 14:12:12.739963 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:12.739555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn" event={"ID":"1a887c26-971a-4e90-b7b5-2ca6e1764f99","Type":"ContainerStarted","Data":"482e4102ad90c073aba16c1ebba3f2d7584585c5743b2e8a588906f430ffb8a6"}
Sep 11 14:12:13.881648 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:13.881618    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"
Sep 11 14:12:14.068857 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:14.068738    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a887c26-971a-4e90-b7b5-2ca6e1764f99-maas-api-service-ca\") pod \"1a887c26-971a-4e90-b7b5-2ca6e1764f99\" (UID: \"1a887c26-971a-4e90-b7b5-2ca6e1764f99\") "
Sep 11 14:12:14.069131 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:14.069108    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a887c26-971a-4e90-b7b5-2ca6e1764f99-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a887c26-971a-4e90-b7b5-2ca6e1764f99" (UID: "1a887c26-971a-4e90-b7b5-2ca6e1764f99"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:12:14.170241 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:14.170186    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a887c26-971a-4e90-b7b5-2ca6e1764f99-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:14.753037 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:14.753011    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"
Sep 11 14:12:14.753239 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:14.753004    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn" event={"ID":"1a887c26-971a-4e90-b7b5-2ca6e1764f99","Type":"ContainerDied","Data":"482e4102ad90c073aba16c1ebba3f2d7584585c5743b2e8a588906f430ffb8a6"}
Sep 11 14:12:14.753239 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:14.753136    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="482e4102ad90c073aba16c1ebba3f2d7584585c5743b2e8a588906f430ffb8a6"
Sep 11 14:12:26.811633 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:26.811598    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"]
Sep 11 14:12:26.812185 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:26.811926    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg" podUID="11668ef3-730c-48c5-bb8c-dc370f19c750" containerName="payload-processing" containerID="cri-o://ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce" gracePeriod=30
Sep 11 14:12:26.994314 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:26.994280    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"]
Sep 11 14:12:26.996766 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:26.996729    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1f2b49-ffrcn"]
Sep 11 14:12:27.082249 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.082223    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:12:27.197204 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.197168    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8k2s\" (UniqueName: \"kubernetes.io/projected/11668ef3-730c-48c5-bb8c-dc370f19c750-kube-api-access-p8k2s\") pod \"11668ef3-730c-48c5-bb8c-dc370f19c750\" (UID: \"11668ef3-730c-48c5-bb8c-dc370f19c750\") "
Sep 11 14:12:27.197414 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.197238    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11668ef3-730c-48c5-bb8c-dc370f19c750-plugins-config-volume\") pod \"11668ef3-730c-48c5-bb8c-dc370f19c750\" (UID: \"11668ef3-730c-48c5-bb8c-dc370f19c750\") "
Sep 11 14:12:27.197611 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.197586    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11668ef3-730c-48c5-bb8c-dc370f19c750-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11668ef3-730c-48c5-bb8c-dc370f19c750" (UID: "11668ef3-730c-48c5-bb8c-dc370f19c750"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:12:27.199660 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.199631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11668ef3-730c-48c5-bb8c-dc370f19c750-kube-api-access-p8k2s" (OuterVolumeSpecName: "kube-api-access-p8k2s") pod "11668ef3-730c-48c5-bb8c-dc370f19c750" (UID: "11668ef3-730c-48c5-bb8c-dc370f19c750"). InnerVolumeSpecName "kube-api-access-p8k2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:27.298047 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.298009    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8k2s\" (UniqueName: \"kubernetes.io/projected/11668ef3-730c-48c5-bb8c-dc370f19c750-kube-api-access-p8k2s\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:27.298047 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.298039    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11668ef3-730c-48c5-bb8c-dc370f19c750-plugins-config-volume\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:27.837662 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.837628    2563 generic.go:356] "Generic (PLEG): container finished" podID="11668ef3-730c-48c5-bb8c-dc370f19c750" containerID="ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce" exitCode=0
Sep 11 14:12:27.838118 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.837690    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"
Sep 11 14:12:27.838118 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.837723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg" event={"ID":"11668ef3-730c-48c5-bb8c-dc370f19c750","Type":"ContainerDied","Data":"ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce"}
Sep 11 14:12:27.838118 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.837829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg" event={"ID":"11668ef3-730c-48c5-bb8c-dc370f19c750","Type":"ContainerDied","Data":"5cf0dfcae7c0fc3e0f0c37d3c449ace616a0fb04bae926862b90e42c1232cf56"}
Sep 11 14:12:27.838118 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.837854    2563 scope.go:117] "RemoveContainer" containerID="ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce"
Sep 11 14:12:27.848005 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.847983    2563 scope.go:117] "RemoveContainer" containerID="ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce"
Sep 11 14:12:27.848308 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:12:27.848288    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce\": container with ID starting with ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce not found: ID does not exist" containerID="ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce"
Sep 11 14:12:27.848389 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.848322    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce"} err="failed to get container status \"ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce\": rpc error: code = NotFound desc = could not find container \"ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce\": container with ID starting with ac8e70974eeef23246d290eb43e14032c6dbb97bfdc7b97966989c16961449ce not found: ID does not exist"
Sep 11 14:12:27.860247 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.860214    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"]
Sep 11 14:12:27.866002 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:27.865965    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f2b49-dc5dcb94f-v4phg"]
Sep 11 14:12:28.240642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.240610    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11668ef3-730c-48c5-bb8c-dc370f19c750" path="/var/lib/kubelet/pods/11668ef3-730c-48c5-bb8c-dc370f19c750/volumes"
Sep 11 14:12:28.241017 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.241003    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a887c26-971a-4e90-b7b5-2ca6e1764f99" path="/var/lib/kubelet/pods/1a887c26-971a-4e90-b7b5-2ca6e1764f99/volumes"
Sep 11 14:12:28.964912 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.964869    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"]
Sep 11 14:12:28.965465 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.965445    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a887c26-971a-4e90-b7b5-2ca6e1764f99" containerName="revoke-keys"
Sep 11 14:12:28.965548 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.965469    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a887c26-971a-4e90-b7b5-2ca6e1764f99" containerName="revoke-keys"
Sep 11 14:12:28.965548 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.965506    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11668ef3-730c-48c5-bb8c-dc370f19c750" containerName="payload-processing"
Sep 11 14:12:28.965548 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.965518    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="11668ef3-730c-48c5-bb8c-dc370f19c750" containerName="payload-processing"
Sep 11 14:12:28.965700 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.965614    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a887c26-971a-4e90-b7b5-2ca6e1764f99" containerName="revoke-keys"
Sep 11 14:12:28.965700 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.965627    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="11668ef3-730c-48c5-bb8c-dc370f19c750" containerName="payload-processing"
Sep 11 14:12:28.970135 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.970112    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:28.974358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.973931    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"kube-root-ca.crt\""
Sep 11 14:12:28.974358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.974079    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"openshift-service-ca.crt\""
Sep 11 14:12:28.974358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.974196    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"rate-test-model-367517-kserve-self-signed-certs\""
Sep 11 14:12:28.974358 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.974251    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-367517\"/\"default-dockercfg-gwmlw\""
Sep 11 14:12:28.977344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:28.977304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"]
Sep 11 14:12:29.015301 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.015266    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-model-cache\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.015485 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.015335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-dshm\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.015485 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.015357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pfnf\" (UniqueName: \"kubernetes.io/projected/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-kube-api-access-5pfnf\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.015485 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.015387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tmp-dir\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.015485 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.015429    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-home\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.015637 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.015494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tls-certs\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116113 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-model-cache\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-dshm\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pfnf\" (UniqueName: \"kubernetes.io/projected/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-kube-api-access-5pfnf\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tmp-dir\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-home\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tls-certs\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116655 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-model-cache\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116655 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tmp-dir\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.116807 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.116658    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-home\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.118585 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.118565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-dshm\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.118932 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.118916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tls-certs\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.128426 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.128404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pfnf\" (UniqueName: \"kubernetes.io/projected/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-kube-api-access-5pfnf\") pod \"rate-test-model-367517-kserve-7f7dc5565f-n274k\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") " pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.280312 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.280214    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:29.562446 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.562419    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"]
Sep 11 14:12:29.564846 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:12:29.564805    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4b1f8af_10d7_4e60_b010_7960b7ae30b0.slice/crio-1065f5444a943bb547873795648de499998948a70806549bb7258a42754041d9 WatchSource:0}: Error finding container 1065f5444a943bb547873795648de499998948a70806549bb7258a42754041d9: Status 404 returned error can't find the container with id 1065f5444a943bb547873795648de499998948a70806549bb7258a42754041d9
Sep 11 14:12:29.853078 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.853038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k" event={"ID":"a4b1f8af-10d7-4e60-b010-7960b7ae30b0","Type":"ContainerStarted","Data":"8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda"}
Sep 11 14:12:29.853257 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.853089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k" event={"ID":"a4b1f8af-10d7-4e60-b010-7960b7ae30b0","Type":"ContainerStarted","Data":"1065f5444a943bb547873795648de499998948a70806549bb7258a42754041d9"}
Sep 11 14:12:29.871049 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:29.871002    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k" podStartSLOduration=1.870984539 podStartE2EDuration="1.870984539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:12:29.869661223 +0000 UTC m=+1446.127861245" watchObservedRunningTime="2026-09-11 14:12:29.870984539 +0000 UTC m=+1446.129184547"
Sep 11 14:12:32.939076 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:32.939030    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:12:34.844408 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:34.844367    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"]
Sep 11 14:12:34.844954 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:34.844601    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy" containerID="cri-o://5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b" gracePeriod=30
Sep 11 14:12:39.280924 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:39.280882    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:39.281355 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:39.280947    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:39.293991 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:39.293961    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:39.930443 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:39.930416    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:12:40.103739 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.103714    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:12:40.226324 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226222    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-socket\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.226324 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226275    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-podinfo\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.226577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxg2m\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-kube-api-access-vxg2m\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.226577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226362    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-certs\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.226577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226388    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-token\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.226577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226436    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-envoy\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.226577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226479    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bb1953c-03b3-4f4b-94de-0d18316738c7-istiod-ca-cert\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.226577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226512    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-data\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.226577 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226563    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:40.226955 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-credential-socket\") pod \"4bb1953c-03b3-4f4b-94de-0d18316738c7\" (UID: \"4bb1953c-03b3-4f4b-94de-0d18316738c7\") "
Sep 11 14:12:40.227011 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.226984    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:40.227070 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.227019    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.227129 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.227093    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:40.227269 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.227244    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:40.227866 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.227838    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb1953c-03b3-4f4b-94de-0d18316738c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:12:40.228939 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.228908    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:12:40.228939 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.228927    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-kube-api-access-vxg2m" (OuterVolumeSpecName: "kube-api-access-vxg2m") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "kube-api-access-vxg2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:40.229123 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.228975    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:40.229184 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.229154    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4bb1953c-03b3-4f4b-94de-0d18316738c7" (UID: "4bb1953c-03b3-4f4b-94de-0d18316738c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:40.327953 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.327905    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxg2m\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-kube-api-access-vxg2m\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.327953 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.327940    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.327953 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.327956    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.328415 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.327966    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.328415 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.327977    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bb1953c-03b3-4f4b-94de-0d18316738c7-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.328415 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.327989    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.328415 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.327999    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bb1953c-03b3-4f4b-94de-0d18316738c7-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.328415 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.328010    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bb1953c-03b3-4f4b-94de-0d18316738c7-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:12:40.927884 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.927844    2563 generic.go:356] "Generic (PLEG): container finished" podID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerID="5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b" exitCode=0
Sep 11 14:12:40.928110 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.927907    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"
Sep 11 14:12:40.928110 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.927940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" event={"ID":"4bb1953c-03b3-4f4b-94de-0d18316738c7","Type":"ContainerDied","Data":"5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b"}
Sep 11 14:12:40.928110 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.928002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f" event={"ID":"4bb1953c-03b3-4f4b-94de-0d18316738c7","Type":"ContainerDied","Data":"8376d77477e4f2f9684dbddbefa7a2bf3fef760bc14eb2796fe539f58744a350"}
Sep 11 14:12:40.928110 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.928033    2563 scope.go:117] "RemoveContainer" containerID="5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b"
Sep 11 14:12:40.938453 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.938429    2563 scope.go:117] "RemoveContainer" containerID="aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb"
Sep 11 14:12:40.948812 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.948726    2563 scope.go:117] "RemoveContainer" containerID="5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b"
Sep 11 14:12:40.949121 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:12:40.949100    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b\": container with ID starting with 5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b not found: ID does not exist" containerID="5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b"
Sep 11 14:12:40.949184 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.949132    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b"} err="failed to get container status \"5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b\": rpc error: code = NotFound desc = could not find container \"5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b\": container with ID starting with 5a5f94769d43b0dc5050cb8809fb31e12bd73ce8b86615824229fcfd26b1fe8b not found: ID does not exist"
Sep 11 14:12:40.949184 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.949149    2563 scope.go:117] "RemoveContainer" containerID="aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb"
Sep 11 14:12:40.949447 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:12:40.949430    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb\": container with ID starting with aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb not found: ID does not exist" containerID="aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb"
Sep 11 14:12:40.949504 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.949450    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb"} err="failed to get container status \"aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb\": rpc error: code = NotFound desc = could not find container \"aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb\": container with ID starting with aacbca51b7bfa726aff1ea2314ab53e871cc86c4c9d7890523b8862993fff9eb not found: ID does not exist"
Sep 11 14:12:40.950333 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.950300    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"]
Sep 11 14:12:40.953332 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:40.953301    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1f2b49-openshift-default-84766cd698-7ln5f"]
Sep 11 14:12:42.241472 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:42.241434    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" path="/var/lib/kubelet/pods/4bb1953c-03b3-4f4b-94de-0d18316738c7/volumes"
Sep 11 14:12:52.835642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:12:52.835604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:13:03.449719 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:03.449668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:13:10.835243 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:10.835204    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"]
Sep 11 14:13:10.835808 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:10.835498    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy" containerID="cri-o://87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7" gracePeriod=30
Sep 11 14:13:16.114267 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.114234    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:13:16.164374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-token\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.164586 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164398    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-credential-socket\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.164586 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164500    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-podinfo\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.164724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164603    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-envoy\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.164724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164636    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-socket\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.164724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164677    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istiod-ca-cert\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.164724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:16.165013 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164726    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-certs\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.165013 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164764    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-data\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.165013 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.164803    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ffsj\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-kube-api-access-2ffsj\") pod \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\" (UID: \"6382874f-b48e-4fd2-b7c4-c0745a559a7a\") "
Sep 11 14:13:16.165013 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.165001    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:16.165237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.165129    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:16.165237 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.165143    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:13:16.165394 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.165358    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:16.165394 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.165380    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:16.165549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.165388    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:16.165549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.165397    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:16.165549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.165440    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:16.167625 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.167596    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:16.167741 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.167633    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:16.167891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.167865    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-kube-api-access-2ffsj" (OuterVolumeSpecName: "kube-api-access-2ffsj") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "kube-api-access-2ffsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:16.168084 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.168060    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6382874f-b48e-4fd2-b7c4-c0745a559a7a" (UID: "6382874f-b48e-4fd2-b7c4-c0745a559a7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:13:16.170823 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.170791    2563 generic.go:356] "Generic (PLEG): container finished" podID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerID="87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7" exitCode=0
Sep 11 14:13:16.170961 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.170846    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"
Sep 11 14:13:16.170961 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.170874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" event={"ID":"6382874f-b48e-4fd2-b7c4-c0745a559a7a","Type":"ContainerDied","Data":"87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7"}
Sep 11 14:13:16.170961 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.170924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x" event={"ID":"6382874f-b48e-4fd2-b7c4-c0745a559a7a","Type":"ContainerDied","Data":"8ff66e982b41d8952ae09204877b7614768a4ccf79235d878dacc7ceaa25bb1e"}
Sep 11 14:13:16.170961 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.170948    2563 scope.go:117] "RemoveContainer" containerID="87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7"
Sep 11 14:13:16.189966 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.189944    2563 scope.go:117] "RemoveContainer" containerID="fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d"
Sep 11 14:13:16.200338 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.200316    2563 scope.go:117] "RemoveContainer" containerID="87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7"
Sep 11 14:13:16.200679 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:13:16.200657    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7\": container with ID starting with 87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7 not found: ID does not exist" containerID="87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7"
Sep 11 14:13:16.200768 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.200688    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7"} err="failed to get container status \"87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7\": rpc error: code = NotFound desc = could not find container \"87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7\": container with ID starting with 87ae141d6c5ba8a539e446abc917244a17ea39af62e1e2b6dbb2a610fa6739b7 not found: ID does not exist"
Sep 11 14:13:16.200768 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.200706    2563 scope.go:117] "RemoveContainer" containerID="fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d"
Sep 11 14:13:16.201037 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:13:16.201017    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d\": container with ID starting with fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d not found: ID does not exist" containerID="fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d"
Sep 11 14:13:16.201096 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.201044    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d"} err="failed to get container status \"fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d\": rpc error: code = NotFound desc = could not find container \"fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d\": container with ID starting with fb3ac64546ea16c039d232e04ea511acdaa009639275610ce500b58641b8cf5d not found: ID does not exist"
Sep 11 14:13:16.201161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.201103    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"]
Sep 11 14:13:16.204870 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.204844    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-01faa9-openshift-default-78d947f544-9f94x"]
Sep 11 14:13:16.241069 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.241032    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" path="/var/lib/kubelet/pods/6382874f-b48e-4fd2-b7c4-c0745a559a7a/volumes"
Sep 11 14:13:16.267118 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.267079    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:16.267118 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.267116    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ffsj\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-kube-api-access-2ffsj\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:16.267337 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.267132    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:16.267337 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.267148    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:16.267337 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:16.267162    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6382874f-b48e-4fd2-b7c4-c0745a559a7a-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:17.446869 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:17.446835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:13:21.304912 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:21.304869    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"]
Sep 11 14:13:22.267201 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:22.267161    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"]
Sep 11 14:13:24.400257 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:24.400228    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:13:24.404601 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:24.404575    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:13:24.565593 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:24.565545    2563 scope.go:117] "RemoveContainer" containerID="636c7521ec6dc6a356ba14eb441f21bee03f92fe4c85f45fd94a263b4a4f6b8a"
Sep 11 14:13:26.791273 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:26.791241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:13:36.325298 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.325246    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4" podUID="2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" containerName="main" containerID="cri-o://26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548" gracePeriod=45
Sep 11 14:13:36.579079 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.579011    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:13:36.672021 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.671988    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlb7d\" (UniqueName: \"kubernetes.io/projected/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-kube-api-access-qlb7d\") pod \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") "
Sep 11 14:13:36.672021 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672030    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tls-certs\") pod \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") "
Sep 11 14:13:36.672332 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672061    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tmp-dir\") pod \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") "
Sep 11 14:13:36.672332 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672116    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-home\") pod \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") "
Sep 11 14:13:36.672332 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672181    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-model-cache\") pod \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") "
Sep 11 14:13:36.672332 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672245    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-dshm\") pod \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\" (UID: \"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8\") "
Sep 11 14:13:36.672549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" (UID: "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:36.672549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672447    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-home" (OuterVolumeSpecName: "home") pod "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" (UID: "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:36.672549 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672463    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" (UID: "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:36.672672 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672644    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:36.672672 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672658    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:36.672672 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.672670    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:36.674480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.674452    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" (UID: "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:13:36.674480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.674454    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-dshm" (OuterVolumeSpecName: "dshm") pod "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" (UID: "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:36.674480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.674474    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-kube-api-access-qlb7d" (OuterVolumeSpecName: "kube-api-access-qlb7d") pod "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" (UID: "2550d4ba-6e9a-49a9-bc17-959b42ebb4d8"). InnerVolumeSpecName "kube-api-access-qlb7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:36.774192 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.774157    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:36.774192 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.774185    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlb7d\" (UniqueName: \"kubernetes.io/projected/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-kube-api-access-qlb7d\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:36.774192 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:36.774196    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:37.287374 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.287331    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k" podUID="a4b1f8af-10d7-4e60-b010-7960b7ae30b0" containerName="main" containerID="cri-o://8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda" gracePeriod=45
Sep 11 14:13:37.305781 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.305720    2563 generic.go:356] "Generic (PLEG): container finished" podID="2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" containerID="26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548" exitCode=0
Sep 11 14:13:37.305944 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.305784    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"
Sep 11 14:13:37.305944 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.305791    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4" event={"ID":"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8","Type":"ContainerDied","Data":"26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548"}
Sep 11 14:13:37.305944 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.305840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4" event={"ID":"2550d4ba-6e9a-49a9-bc17-959b42ebb4d8","Type":"ContainerDied","Data":"4e9dd7fb3952e01d1ba98ea4dae9ae41995e060e02695822412bef60e153da49"}
Sep 11 14:13:37.305944 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.305859    2563 scope.go:117] "RemoveContainer" containerID="26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548"
Sep 11 14:13:37.363896 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.363855    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"]
Sep 11 14:13:37.367122 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.367092    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-932c97/rate-test-model-932c97-kserve-6d59f56d99-5bjv4"]
Sep 11 14:13:37.399716 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.399523    2563 scope.go:117] "RemoveContainer" containerID="26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548"
Sep 11 14:13:37.399944 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:13:37.399924    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548\": container with ID starting with 26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548 not found: ID does not exist" containerID="26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548"
Sep 11 14:13:37.400011 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.399961    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548"} err="failed to get container status \"26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548\": rpc error: code = NotFound desc = could not find container \"26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548\": container with ID starting with 26cc73eab9701d92a552cc2eff316aa63bd291dffc208566bdc7577707ffa548 not found: ID does not exist"
Sep 11 14:13:37.558147 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.558118    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:13:37.683661 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.683616    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-home\") pod \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") "
Sep 11 14:13:37.683661 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.683666    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tls-certs\") pod \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") "
Sep 11 14:13:37.683931 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.683735    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-dshm\") pod \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") "
Sep 11 14:13:37.683931 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.683817    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pfnf\" (UniqueName: \"kubernetes.io/projected/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-kube-api-access-5pfnf\") pod \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") "
Sep 11 14:13:37.683931 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.683850    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tmp-dir\") pod \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") "
Sep 11 14:13:37.683931 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.683870    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-model-cache\") pod \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\" (UID: \"a4b1f8af-10d7-4e60-b010-7960b7ae30b0\") "
Sep 11 14:13:37.684144 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.683954    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-home" (OuterVolumeSpecName: "home") pod "a4b1f8af-10d7-4e60-b010-7960b7ae30b0" (UID: "a4b1f8af-10d7-4e60-b010-7960b7ae30b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:37.684219 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.684196    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4b1f8af-10d7-4e60-b010-7960b7ae30b0" (UID: "a4b1f8af-10d7-4e60-b010-7960b7ae30b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:37.684276 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.684214    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4b1f8af-10d7-4e60-b010-7960b7ae30b0" (UID: "a4b1f8af-10d7-4e60-b010-7960b7ae30b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:37.684325 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.684276    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:37.686259 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.686237    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-dshm" (OuterVolumeSpecName: "dshm") pod "a4b1f8af-10d7-4e60-b010-7960b7ae30b0" (UID: "a4b1f8af-10d7-4e60-b010-7960b7ae30b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:13:37.686504 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.686487    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4b1f8af-10d7-4e60-b010-7960b7ae30b0" (UID: "a4b1f8af-10d7-4e60-b010-7960b7ae30b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:13:37.686593 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.686573    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-kube-api-access-5pfnf" (OuterVolumeSpecName: "kube-api-access-5pfnf") pod "a4b1f8af-10d7-4e60-b010-7960b7ae30b0" (UID: "a4b1f8af-10d7-4e60-b010-7960b7ae30b0"). InnerVolumeSpecName "kube-api-access-5pfnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:37.785135 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.785085    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:37.785135 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.785133    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pfnf\" (UniqueName: \"kubernetes.io/projected/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-kube-api-access-5pfnf\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:37.785135 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.785147    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:37.785135 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.785157    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:37.785448 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:37.785165    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b1f8af-10d7-4e60-b010-7960b7ae30b0-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:13:38.241610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.241577    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" path="/var/lib/kubelet/pods/2550d4ba-6e9a-49a9-bc17-959b42ebb4d8/volumes"
Sep 11 14:13:38.314434 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.314398    2563 generic.go:356] "Generic (PLEG): container finished" podID="a4b1f8af-10d7-4e60-b010-7960b7ae30b0" containerID="8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda" exitCode=0
Sep 11 14:13:38.314642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.314441    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"
Sep 11 14:13:38.314642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.314485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k" event={"ID":"a4b1f8af-10d7-4e60-b010-7960b7ae30b0","Type":"ContainerDied","Data":"8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda"}
Sep 11 14:13:38.314642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.314529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k" event={"ID":"a4b1f8af-10d7-4e60-b010-7960b7ae30b0","Type":"ContainerDied","Data":"1065f5444a943bb547873795648de499998948a70806549bb7258a42754041d9"}
Sep 11 14:13:38.314642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.314552    2563 scope.go:117] "RemoveContainer" containerID="8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda"
Sep 11 14:13:38.324163 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.324144    2563 scope.go:117] "RemoveContainer" containerID="8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda"
Sep 11 14:13:38.324488 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:13:38.324469    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda\": container with ID starting with 8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda not found: ID does not exist" containerID="8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda"
Sep 11 14:13:38.324544 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.324497    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda"} err="failed to get container status \"8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda\": rpc error: code = NotFound desc = could not find container \"8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda\": container with ID starting with 8a544976d27d880c2f0e73ef76bf738e989fa3c1a7d534081776eb0faeb13cda not found: ID does not exist"
Sep 11 14:13:38.331685 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.331655    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"]
Sep 11 14:13:38.335740 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:38.335709    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-367517/rate-test-model-367517-kserve-7f7dc5565f-n274k"]
Sep 11 14:13:40.149279 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:40.149241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:13:40.241043 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:40.241009    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4b1f8af-10d7-4e60-b010-7960b7ae30b0" path="/var/lib/kubelet/pods/a4b1f8af-10d7-4e60-b010-7960b7ae30b0/volumes"
Sep 11 14:13:52.083096 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083051    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"]
Sep 11 14:13:52.083660 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083639    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083664    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083681    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083690    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083706    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b1f8af-10d7-4e60-b010-7960b7ae30b0" containerName="main"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083714    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b1f8af-10d7-4e60-b010-7960b7ae30b0" containerName="main"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083732    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083740    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083769    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" containerName="main"
Sep 11 14:13:52.083787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083778    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" containerName="main"
Sep 11 14:13:52.084232 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083794    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy"
Sep 11 14:13:52.084232 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083802    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy"
Sep 11 14:13:52.084232 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083901    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy"
Sep 11 14:13:52.084232 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083916    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2550d4ba-6e9a-49a9-bc17-959b42ebb4d8" containerName="main"
Sep 11 14:13:52.084232 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083926    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy"
Sep 11 14:13:52.084232 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083937    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bb1953c-03b3-4f4b-94de-0d18316738c7" containerName="istio-proxy"
Sep 11 14:13:52.084232 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083949    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4b1f8af-10d7-4e60-b010-7960b7ae30b0" containerName="main"
Sep 11 14:13:52.084232 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.083959    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6382874f-b48e-4fd2-b7c4-c0745a559a7a" containerName="istio-proxy"
Sep 11 14:13:52.089037 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.089011    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.092418 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.092387    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-68d0c7-openshift-default-dockercfg-krvdw\""
Sep 11 14:13:52.105194 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.105157    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"]
Sep 11 14:13:52.122006 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.121971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-data\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.122006 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.122013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-envoy\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.122006 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.122029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-podinfo\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.122351 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.122045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-certs\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.122351 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.122114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-token\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.122351 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.122145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-credential-socket\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.122351 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.122171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccc2735-0e68-4a09-9d6b-5668c5544790-istiod-ca-cert\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.122351 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.122254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlflz\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-kube-api-access-mlflz\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.122351 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.122289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-socket\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223273 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223225    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-data\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223273 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-envoy\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223541 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-podinfo\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223541 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-certs\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223541 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-token\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223541 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-credential-socket\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223792 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccc2735-0e68-4a09-9d6b-5668c5544790-istiod-ca-cert\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223792 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223690    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlflz\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-kube-api-access-mlflz\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.223792 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.223718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-socket\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.224111 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.224054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-data\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.224111 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.224108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-credential-socket\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.224304 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.224273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-socket\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.224379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.224355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-certs\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.224468 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.224451    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccc2735-0e68-4a09-9d6b-5668c5544790-istiod-ca-cert\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.225908 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.225889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-envoy\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.226051 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.226034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-podinfo\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.232546 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.232515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-token\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.232660 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.232636    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlflz\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-kube-api-access-mlflz\") pod \"e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") " pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.400911 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.400820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:52.686353 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.686317    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"]
Sep 11 14:13:52.688791 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:13:52.688735    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccc2735_0e68_4a09_9d6b_5668c5544790.slice/crio-b5d17361219a75adb166c0c8d6f0c00d74446cc9f0018946b8fa01f1885c426f WatchSource:0}: Error finding container b5d17361219a75adb166c0c8d6f0c00d74446cc9f0018946b8fa01f1885c426f: Status 404 returned error can't find the container with id b5d17361219a75adb166c0c8d6f0c00d74446cc9f0018946b8fa01f1885c426f
Sep 11 14:13:52.691380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.691345    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:52.691480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.691426    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:52.691480 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:52.691456    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:53.408871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:53.408831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" event={"ID":"eccc2735-0e68-4a09-9d6b-5668c5544790","Type":"ContainerStarted","Data":"e5d28542dd97265bab117b767f5d6de74db6ed7540584022bf238e6a665597f4"}
Sep 11 14:13:53.409384 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:53.408892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" event={"ID":"eccc2735-0e68-4a09-9d6b-5668c5544790","Type":"ContainerStarted","Data":"b5d17361219a75adb166c0c8d6f0c00d74446cc9f0018946b8fa01f1885c426f"}
Sep 11 14:13:53.430046 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:53.429989    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" podStartSLOduration=1.429971178 podStartE2EDuration="1.429971178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:53.428233435 +0000 UTC m=+1529.686433446" watchObservedRunningTime="2026-09-11 14:13:53.429971178 +0000 UTC m=+1529.688171187"
Sep 11 14:13:54.401874 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:54.401834    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:54.408178 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:54.408146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:54.414898 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:54.414867    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:54.415734 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:54.415715    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:13:57.237142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.237105    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"]
Sep 11 14:13:57.240509 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.240484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:57.243301 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.243270    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-68d0c7-dockercfg-72526\""
Sep 11 14:13:57.243443 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.243338    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-68d0c7\""
Sep 11 14:13:57.258814 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.258775    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"]
Sep 11 14:13:57.274457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.274416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab30bdb-6b4c-404a-a259-6b185a77aa8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx\" (UID: \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:57.274609 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.274526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck7vq\" (UniqueName: \"kubernetes.io/projected/fab30bdb-6b4c-404a-a259-6b185a77aa8f-kube-api-access-ck7vq\") pod \"payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx\" (UID: \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:57.375728 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.375688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab30bdb-6b4c-404a-a259-6b185a77aa8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx\" (UID: \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:57.375728 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.375776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck7vq\" (UniqueName: \"kubernetes.io/projected/fab30bdb-6b4c-404a-a259-6b185a77aa8f-kube-api-access-ck7vq\") pod \"payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx\" (UID: \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:57.376586 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.376565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab30bdb-6b4c-404a-a259-6b185a77aa8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx\" (UID: \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:57.386422 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.386392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck7vq\" (UniqueName: \"kubernetes.io/projected/fab30bdb-6b4c-404a-a259-6b185a77aa8f-kube-api-access-ck7vq\") pod \"payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx\" (UID: \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:57.552686 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.552579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:57.830973 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:57.830946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"]
Sep 11 14:13:57.832956 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:13:57.832928    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab30bdb_6b4c_404a_a259_6b185a77aa8f.slice/crio-98755ce2db519fe45cb6e3f09a77a2b07d67b5c766cb6d9ce505a9595075f07f WatchSource:0}: Error finding container 98755ce2db519fe45cb6e3f09a77a2b07d67b5c766cb6d9ce505a9595075f07f: Status 404 returned error can't find the container with id 98755ce2db519fe45cb6e3f09a77a2b07d67b5c766cb6d9ce505a9595075f07f
Sep 11 14:13:58.441057 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:58.441022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx" event={"ID":"fab30bdb-6b4c-404a-a259-6b185a77aa8f","Type":"ContainerStarted","Data":"adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3"}
Sep 11 14:13:58.441458 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:58.441076    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx" event={"ID":"fab30bdb-6b4c-404a-a259-6b185a77aa8f","Type":"ContainerStarted","Data":"98755ce2db519fe45cb6e3f09a77a2b07d67b5c766cb6d9ce505a9595075f07f"}
Sep 11 14:13:58.441458 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:58.441260    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:13:58.459369 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:13:58.459313    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx" podStartSLOduration=1.459293321 podStartE2EDuration="1.459293321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:58.456901457 +0000 UTC m=+1534.715101468" watchObservedRunningTime="2026-09-11 14:13:58.459293321 +0000 UTC m=+1534.717493330"
Sep 11 14:14:02.644704 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:02.644668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:14:09.449130 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:09.449088    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:14:13.544477 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:13.544374    2563 generic.go:356] "Generic (PLEG): container finished" podID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerID="e5d28542dd97265bab117b767f5d6de74db6ed7540584022bf238e6a665597f4" exitCode=0
Sep 11 14:14:13.544477 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:13.544449    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" event={"ID":"eccc2735-0e68-4a09-9d6b-5668c5544790","Type":"ContainerDied","Data":"e5d28542dd97265bab117b767f5d6de74db6ed7540584022bf238e6a665597f4"}
Sep 11 14:14:13.544957 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:13.544900    2563 scope.go:117] "RemoveContainer" containerID="e5d28542dd97265bab117b767f5d6de74db6ed7540584022bf238e6a665597f4"
Sep 11 14:14:13.546071 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:13.546040    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:14:13.546178 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:13.546105    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:14:13.546178 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:13.546139    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:14:14.552836 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:14.552800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" event={"ID":"eccc2735-0e68-4a09-9d6b-5668c5544790","Type":"ContainerStarted","Data":"4544c7a5b896d0c8d5c23e8465d7a85f6438386e0b70961a02da9e3cd5f15084"}
Sep 11 14:14:15.402344 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:15.402293    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:14:15.402590 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:15.402442    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 11 14:14:15.402590 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:15.402495    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 11 14:14:16.401545 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:16.401508    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 11 14:14:16.402089 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:16.401566    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 11 14:14:17.401951 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:17.401904    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 11 14:14:17.402492 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:17.401979    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 11 14:14:18.401849 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:18.401797    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 11 14:14:18.402076 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:18.401878    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 11 14:14:19.402558 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:19.402517    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 11 14:14:19.403651 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:19.403223    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 11 14:14:20.406359 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:20.406329    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:14:20.406821 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:20.406604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:14:20.407314 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:20.407297    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:14:36.460565 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.460520    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"]
Sep 11 14:14:36.464833 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.464805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.468663 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.468033    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-e3622e\"/\"openshift-service-ca.crt\""
Sep 11 14:14:36.468663 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.468135    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-e3622e\"/\"kube-root-ca.crt\""
Sep 11 14:14:36.468663 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.468045    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-e3622e\"/\"ipp-route-e3622e-kserve-self-signed-certs\""
Sep 11 14:14:36.468663 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.468401    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-e3622e\"/\"default-dockercfg-mpwrb\""
Sep 11 14:14:36.470019 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.469988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/108876d6-417e-45a1-be58-1f7d7109eaba-tls-certs\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.470430 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.470402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t6tz\" (UniqueName: \"kubernetes.io/projected/108876d6-417e-45a1-be58-1f7d7109eaba-kube-api-access-8t6tz\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.470620 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.470594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-home\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.470730 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.470660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-model-cache\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.470730 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.470683    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-tmp-dir\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.470882 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.470854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-dshm\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.470936 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.470900    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"]
Sep 11 14:14:36.572226 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/108876d6-417e-45a1-be58-1f7d7109eaba-tls-certs\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.572226 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t6tz\" (UniqueName: \"kubernetes.io/projected/108876d6-417e-45a1-be58-1f7d7109eaba-kube-api-access-8t6tz\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.572504 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-home\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.572504 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-model-cache\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.572504 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-tmp-dir\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.572504 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-dshm\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.572771 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-home\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.572894 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-model-cache\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.572894 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.572814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-tmp-dir\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.574877 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.574855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-dshm\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.575286 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.575265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/108876d6-417e-45a1-be58-1f7d7109eaba-tls-certs\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.580742 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.580714    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t6tz\" (UniqueName: \"kubernetes.io/projected/108876d6-417e-45a1-be58-1f7d7109eaba-kube-api-access-8t6tz\") pod \"ipp-route-e3622e-kserve-6cb886bdd7-p8bvt\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") " pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:36.779163 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:36.779067    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:37.055432 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:37.055405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"]
Sep 11 14:14:37.057180 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:14:37.057151    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod108876d6_417e_45a1_be58_1f7d7109eaba.slice/crio-70213913defe2496c3bb543c6d78892074235c824d562ae07a5b680137c1b55b WatchSource:0}: Error finding container 70213913defe2496c3bb543c6d78892074235c824d562ae07a5b680137c1b55b: Status 404 returned error can't find the container with id 70213913defe2496c3bb543c6d78892074235c824d562ae07a5b680137c1b55b
Sep 11 14:14:37.695504 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:37.695466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt" event={"ID":"108876d6-417e-45a1-be58-1f7d7109eaba","Type":"ContainerStarted","Data":"3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f"}
Sep 11 14:14:37.695504 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:37.695510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt" event={"ID":"108876d6-417e-45a1-be58-1f7d7109eaba","Type":"ContainerStarted","Data":"70213913defe2496c3bb543c6d78892074235c824d562ae07a5b680137c1b55b"}
Sep 11 14:14:37.715992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:37.715937    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt" podStartSLOduration=1.715918717 podStartE2EDuration="1.715918717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:14:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:14:37.713107678 +0000 UTC m=+1573.971307687" watchObservedRunningTime="2026-09-11 14:14:37.715918717 +0000 UTC m=+1573.974118728"
Sep 11 14:14:40.634431 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:40.634382    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:14:46.779734 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:46.779694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:46.779734 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:46.779745    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:46.792447 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:46.792418    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:14:47.765425 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:14:47.765399    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:15:00.145672 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.145631    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"]
Sep 11 14:15:00.148571 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.148549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"
Sep 11 14:15:00.149875 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.149850    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"]
Sep 11 14:15:00.151269 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.151250    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:15:00.152171 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.152142    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"
Sep 11 14:15:00.162141 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.162099    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"]
Sep 11 14:15:00.166275 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.166244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"
Sep 11 14:15:00.167898 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.167868    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"]
Sep 11 14:15:00.172341 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.172317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"
Sep 11 14:15:00.449261 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.449226    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"]
Sep 11 14:15:00.451280 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:15:00.451254    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a13829_a586_4b96_98b1_e96b6dbeab85.slice/crio-cea3eb3bf76c03466280bd97d3bbcee1486a06fd9316a2cffb5b7a5ab693753b WatchSource:0}: Error finding container cea3eb3bf76c03466280bd97d3bbcee1486a06fd9316a2cffb5b7a5ab693753b: Status 404 returned error can't find the container with id cea3eb3bf76c03466280bd97d3bbcee1486a06fd9316a2cffb5b7a5ab693753b
Sep 11 14:15:00.466279 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.466251    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"]
Sep 11 14:15:00.468295 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:15:00.468271    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60bf19f_ba44_485d_a027_231054a83805.slice/crio-775646fa924254f607977c7426e8bea7daf93bbdb95f6a6446902debd2c6f287 WatchSource:0}: Error finding container 775646fa924254f607977c7426e8bea7daf93bbdb95f6a6446902debd2c6f287: Status 404 returned error can't find the container with id 775646fa924254f607977c7426e8bea7daf93bbdb95f6a6446902debd2c6f287
Sep 11 14:15:00.834014 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.833978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt" event={"ID":"a60bf19f-ba44-485d-a027-231054a83805","Type":"ContainerStarted","Data":"a41214c7d588c140bbfd8ccb09486e1dfb8e076475fbbf8f987b8605aeb297d3"}
Sep 11 14:15:00.834233 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.834042    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt" event={"ID":"a60bf19f-ba44-485d-a027-231054a83805","Type":"ContainerStarted","Data":"775646fa924254f607977c7426e8bea7daf93bbdb95f6a6446902debd2c6f287"}
Sep 11 14:15:00.835691 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.835665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r" event={"ID":"d0a13829-a586-4b96-98b1-e96b6dbeab85","Type":"ContainerStarted","Data":"d9535a97647c7da7360865ad3d6cce8eaf55d3a10aaac36881dc8e132e8e22b9"}
Sep 11 14:15:00.835849 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.835777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r" event={"ID":"d0a13829-a586-4b96-98b1-e96b6dbeab85","Type":"ContainerStarted","Data":"cea3eb3bf76c03466280bd97d3bbcee1486a06fd9316a2cffb5b7a5ab693753b"}
Sep 11 14:15:00.847787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.847701    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt" podStartSLOduration=0.847681022 podStartE2EDuration="847.681022ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:15:00.847310319 +0000 UTC m=+1597.105510322" watchObservedRunningTime="2026-09-11 14:15:00.847681022 +0000 UTC m=+1597.105881032"
Sep 11 14:15:00.862386 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:00.862330    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r" podStartSLOduration=0.862312907 podStartE2EDuration="862.312907ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:15:00.860350701 +0000 UTC m=+1597.118550707" watchObservedRunningTime="2026-09-11 14:15:00.862312907 +0000 UTC m=+1597.120512916"
Sep 11 14:15:01.844194 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:01.844098    2563 generic.go:356] "Generic (PLEG): container finished" podID="d0a13829-a586-4b96-98b1-e96b6dbeab85" containerID="d9535a97647c7da7360865ad3d6cce8eaf55d3a10aaac36881dc8e132e8e22b9" exitCode=0
Sep 11 14:15:01.844655 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:01.844231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r" event={"ID":"d0a13829-a586-4b96-98b1-e96b6dbeab85","Type":"ContainerDied","Data":"d9535a97647c7da7360865ad3d6cce8eaf55d3a10aaac36881dc8e132e8e22b9"}
Sep 11 14:15:01.845998 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:01.845962    2563 generic.go:356] "Generic (PLEG): container finished" podID="a60bf19f-ba44-485d-a027-231054a83805" containerID="a41214c7d588c140bbfd8ccb09486e1dfb8e076475fbbf8f987b8605aeb297d3" exitCode=0
Sep 11 14:15:01.846100 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:01.846043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt" event={"ID":"a60bf19f-ba44-485d-a027-231054a83805","Type":"ContainerDied","Data":"a41214c7d588c140bbfd8ccb09486e1dfb8e076475fbbf8f987b8605aeb297d3"}
Sep 11 14:15:02.994399 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:02.994370    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"
Sep 11 14:15:03.030266 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:03.030239    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"
Sep 11 14:15:03.868473 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:03.868442    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"
Sep 11 14:15:03.868665 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:03.868482    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r" event={"ID":"d0a13829-a586-4b96-98b1-e96b6dbeab85","Type":"ContainerDied","Data":"cea3eb3bf76c03466280bd97d3bbcee1486a06fd9316a2cffb5b7a5ab693753b"}
Sep 11 14:15:03.868665 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:03.868533    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cea3eb3bf76c03466280bd97d3bbcee1486a06fd9316a2cffb5b7a5ab693753b"
Sep 11 14:15:03.870050 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:03.870027    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"
Sep 11 14:15:03.870177 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:03.870051    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt" event={"ID":"a60bf19f-ba44-485d-a027-231054a83805","Type":"ContainerDied","Data":"775646fa924254f607977c7426e8bea7daf93bbdb95f6a6446902debd2c6f287"}
Sep 11 14:15:03.870177 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:03.870090    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="775646fa924254f607977c7426e8bea7daf93bbdb95f6a6446902debd2c6f287"
Sep 11 14:15:05.333055 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:05.333005    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:15:17.508625 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.508536    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"]
Sep 11 14:15:17.509051 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.509034    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0a13829-a586-4b96-98b1-e96b6dbeab85" containerName="cleanup"
Sep 11 14:15:17.509097 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.509053    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0a13829-a586-4b96-98b1-e96b6dbeab85" containerName="cleanup"
Sep 11 14:15:17.509097 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.509064    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a60bf19f-ba44-485d-a027-231054a83805" containerName="cleanup"
Sep 11 14:15:17.509097 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.509070    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a60bf19f-ba44-485d-a027-231054a83805" containerName="cleanup"
Sep 11 14:15:17.509193 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.509145    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a60bf19f-ba44-485d-a027-231054a83805" containerName="cleanup"
Sep 11 14:15:17.509193 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.509153    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0a13829-a586-4b96-98b1-e96b6dbeab85" containerName="cleanup"
Sep 11 14:15:17.511886 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.511870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:17.514739 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.514715    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-24864d\""
Sep 11 14:15:17.514901 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.514796    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-24864d-dockercfg-kqnnf\""
Sep 11 14:15:17.524687 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.524657    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"]
Sep 11 14:15:17.591777 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.591712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc\" (UID: \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:17.591990 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.591937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntc4d\" (UniqueName: \"kubernetes.io/projected/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-kube-api-access-ntc4d\") pod \"payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc\" (UID: \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:17.693525 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.693478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntc4d\" (UniqueName: \"kubernetes.io/projected/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-kube-api-access-ntc4d\") pod \"payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc\" (UID: \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:17.693843 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.693624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc\" (UID: \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:17.694387 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.694367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc\" (UID: \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:17.702268 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.702244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntc4d\" (UniqueName: \"kubernetes.io/projected/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-kube-api-access-ntc4d\") pod \"payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc\" (UID: \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:17.823506 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:17.823406    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:18.100246 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:18.100218    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"]
Sep 11 14:15:18.101956 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:15:18.101926    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab44c9b1_5d25_41c6_bc2e_8d6f1c1df301.slice/crio-24d8faa149e8614dd320b0451e260fddd6bdc961a517343dc3ac402588e1026d WatchSource:0}: Error finding container 24d8faa149e8614dd320b0451e260fddd6bdc961a517343dc3ac402588e1026d: Status 404 returned error can't find the container with id 24d8faa149e8614dd320b0451e260fddd6bdc961a517343dc3ac402588e1026d
Sep 11 14:15:18.967952 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:18.967911    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc" event={"ID":"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301","Type":"ContainerStarted","Data":"83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0"}
Sep 11 14:15:18.968416 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:18.967968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc" event={"ID":"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301","Type":"ContainerStarted","Data":"24d8faa149e8614dd320b0451e260fddd6bdc961a517343dc3ac402588e1026d"}
Sep 11 14:15:18.968416 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:18.968033    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:18.986223 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:18.986169    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc" podStartSLOduration=1.986151803 podStartE2EDuration="1.986151803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:15:18.984339598 +0000 UTC m=+1615.242539608" watchObservedRunningTime="2026-09-11 14:15:18.986151803 +0000 UTC m=+1615.244351811"
Sep 11 14:15:24.619616 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:24.619583    2563 scope.go:117] "RemoveContainer" containerID="6b6a48137fd34a2713575b5da9f0cbbdb14de67779e892b8a7f094536b0ac171"
Sep 11 14:15:25.140352 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:25.140315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:15:29.977172 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:29.977141    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:47.560013 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:47.559971    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"]
Sep 11 14:15:47.560456 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:47.560244    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc" podUID="ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301" containerName="payload-processing" containerID="cri-o://83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0" gracePeriod=30
Sep 11 14:15:47.825934 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:47.825863    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:47.991882 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:47.991826    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntc4d\" (UniqueName: \"kubernetes.io/projected/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-kube-api-access-ntc4d\") pod \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\" (UID: \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\") "
Sep 11 14:15:47.992079 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:47.991958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-plugins-config-volume\") pod \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\" (UID: \"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301\") "
Sep 11 14:15:47.992319 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:47.992297    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301" (UID: "ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:15:47.994202 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:47.994168    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-kube-api-access-ntc4d" (OuterVolumeSpecName: "kube-api-access-ntc4d") pod "ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301" (UID: "ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301"). InnerVolumeSpecName "kube-api-access-ntc4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:15:48.093203 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.093120    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-plugins-config-volume\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:15:48.093203 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.093152    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntc4d\" (UniqueName: \"kubernetes.io/projected/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301-kube-api-access-ntc4d\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:15:48.150795 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.150739    2563 generic.go:356] "Generic (PLEG): container finished" podID="ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301" containerID="83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0" exitCode=0
Sep 11 14:15:48.150795 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.150786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc" event={"ID":"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301","Type":"ContainerDied","Data":"83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0"}
Sep 11 14:15:48.151016 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.150810    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"
Sep 11 14:15:48.151016 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.150857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc" event={"ID":"ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301","Type":"ContainerDied","Data":"24d8faa149e8614dd320b0451e260fddd6bdc961a517343dc3ac402588e1026d"}
Sep 11 14:15:48.151016 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.150893    2563 scope.go:117] "RemoveContainer" containerID="83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0"
Sep 11 14:15:48.160966 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.160914    2563 scope.go:117] "RemoveContainer" containerID="83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0"
Sep 11 14:15:48.161270 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:15:48.161250    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0\": container with ID starting with 83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0 not found: ID does not exist" containerID="83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0"
Sep 11 14:15:48.161324 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.161277    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0"} err="failed to get container status \"83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0\": rpc error: code = NotFound desc = could not find container \"83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0\": container with ID starting with 83dc2664fe77d6a1661f97c4e3f596ddb8b1202abd6cbb013adffc59eac898d0 not found: ID does not exist"
Sep 11 14:15:48.173656 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.173619    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"]
Sep 11 14:15:48.177203 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.177176    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-24864d-7cd4d5b9dc-572cc"]
Sep 11 14:15:48.241834 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:48.241801    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301" path="/var/lib/kubelet/pods/ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301/volumes"
Sep 11 14:15:51.852398 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:51.852362    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:15:57.898981 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.898950    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"]
Sep 11 14:15:57.899379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.899357    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301" containerName="payload-processing"
Sep 11 14:15:57.899379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.899368    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301" containerName="payload-processing"
Sep 11 14:15:57.899459 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.899448    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab44c9b1-5d25-41c6-bc2e-8d6f1c1df301" containerName="payload-processing"
Sep 11 14:15:57.901477 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.901461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"
Sep 11 14:15:57.904224 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.904202    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmpt\""
Sep 11 14:15:57.904373 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.904205    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:15:57.910290 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.910264    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"]
Sep 11 14:15:57.989702 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:57.989660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/619ba29d-0766-4f2a-8694-d31e3ccc78a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn\" (UID: \"619ba29d-0766-4f2a-8694-d31e3ccc78a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"
Sep 11 14:15:58.090298 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:58.090264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/619ba29d-0766-4f2a-8694-d31e3ccc78a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn\" (UID: \"619ba29d-0766-4f2a-8694-d31e3ccc78a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"
Sep 11 14:15:58.090953 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:58.090933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/619ba29d-0766-4f2a-8694-d31e3ccc78a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn\" (UID: \"619ba29d-0766-4f2a-8694-d31e3ccc78a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"
Sep 11 14:15:58.213419 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:58.213336    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"
Sep 11 14:15:58.487416 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:58.487392    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"]
Sep 11 14:15:58.489559 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:15:58.489523    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619ba29d_0766_4f2a_8694_d31e3ccc78a0.slice/crio-fd8aa5b81e56d2423734dfba97b8eafb1f3cc8e97259fb2ee63a2393850bdebf WatchSource:0}: Error finding container fd8aa5b81e56d2423734dfba97b8eafb1f3cc8e97259fb2ee63a2393850bdebf: Status 404 returned error can't find the container with id fd8aa5b81e56d2423734dfba97b8eafb1f3cc8e97259fb2ee63a2393850bdebf
Sep 11 14:15:59.220578 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:59.220540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn" event={"ID":"619ba29d-0766-4f2a-8694-d31e3ccc78a0","Type":"ContainerStarted","Data":"7df8a43e236638626fa54f248c3d5067ae83448f6aacc0c3c9ba20b3a8ce0be3"}
Sep 11 14:15:59.220998 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:59.220602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn" event={"ID":"619ba29d-0766-4f2a-8694-d31e3ccc78a0","Type":"ContainerStarted","Data":"fd8aa5b81e56d2423734dfba97b8eafb1f3cc8e97259fb2ee63a2393850bdebf"}
Sep 11 14:15:59.236070 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:15:59.236016    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn" podStartSLOduration=2.235998203 podStartE2EDuration="2.235998203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:15:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:15:59.235090573 +0000 UTC m=+1655.493290583" watchObservedRunningTime="2026-09-11 14:15:59.235998203 +0000 UTC m=+1655.494198216"
Sep 11 14:16:00.229424 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:00.229386    2563 generic.go:356] "Generic (PLEG): container finished" podID="619ba29d-0766-4f2a-8694-d31e3ccc78a0" containerID="7df8a43e236638626fa54f248c3d5067ae83448f6aacc0c3c9ba20b3a8ce0be3" exitCode=0
Sep 11 14:16:00.229861 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:00.229472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn" event={"ID":"619ba29d-0766-4f2a-8694-d31e3ccc78a0","Type":"ContainerDied","Data":"7df8a43e236638626fa54f248c3d5067ae83448f6aacc0c3c9ba20b3a8ce0be3"}
Sep 11 14:16:01.378505 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:01.378480    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"
Sep 11 14:16:01.421798 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:01.421746    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/619ba29d-0766-4f2a-8694-d31e3ccc78a0-maas-api-service-ca\") pod \"619ba29d-0766-4f2a-8694-d31e3ccc78a0\" (UID: \"619ba29d-0766-4f2a-8694-d31e3ccc78a0\") "
Sep 11 14:16:01.422094 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:01.422070    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/619ba29d-0766-4f2a-8694-d31e3ccc78a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "619ba29d-0766-4f2a-8694-d31e3ccc78a0" (UID: "619ba29d-0766-4f2a-8694-d31e3ccc78a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:16:01.523299 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:01.523209    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/619ba29d-0766-4f2a-8694-d31e3ccc78a0-maas-api-service-ca\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:16:02.242947 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:02.242865    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"
Sep 11 14:16:02.242947 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:02.242898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn" event={"ID":"619ba29d-0766-4f2a-8694-d31e3ccc78a0","Type":"ContainerDied","Data":"fd8aa5b81e56d2423734dfba97b8eafb1f3cc8e97259fb2ee63a2393850bdebf"}
Sep 11 14:16:02.243192 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:02.242995    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd8aa5b81e56d2423734dfba97b8eafb1f3cc8e97259fb2ee63a2393850bdebf"
Sep 11 14:16:11.031521 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:11.031482    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:16:13.455886 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:13.455849    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"]
Sep 11 14:16:13.461528 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:13.461494    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e3622e-29818935-z87pt"]
Sep 11 14:16:13.687975 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:13.687935    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"]
Sep 11 14:16:13.696004 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:13.695970    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e3622e-868rn"]
Sep 11 14:16:14.240900 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:14.240863    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="619ba29d-0766-4f2a-8694-d31e3ccc78a0" path="/var/lib/kubelet/pods/619ba29d-0766-4f2a-8694-d31e3ccc78a0/volumes"
Sep 11 14:16:14.241179 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:14.241167    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a60bf19f-ba44-485d-a027-231054a83805" path="/var/lib/kubelet/pods/a60bf19f-ba44-485d-a027-231054a83805/volumes"
Sep 11 14:16:19.060952 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:19.060917    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"]
Sep 11 14:16:23.837475 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:23.837428    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:16:34.081926 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.081858    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt" podUID="108876d6-417e-45a1-be58-1f7d7109eaba" containerName="main" containerID="cri-o://3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f" gracePeriod=45
Sep 11 14:16:34.340885 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.340812    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:16:34.350070 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350038    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/108876d6-417e-45a1-be58-1f7d7109eaba-tls-certs\") pod \"108876d6-417e-45a1-be58-1f7d7109eaba\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") "
Sep 11 14:16:34.350274 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-dshm\") pod \"108876d6-417e-45a1-be58-1f7d7109eaba\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") "
Sep 11 14:16:34.350274 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350113    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t6tz\" (UniqueName: \"kubernetes.io/projected/108876d6-417e-45a1-be58-1f7d7109eaba-kube-api-access-8t6tz\") pod \"108876d6-417e-45a1-be58-1f7d7109eaba\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") "
Sep 11 14:16:34.350274 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350144    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-home\") pod \"108876d6-417e-45a1-be58-1f7d7109eaba\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") "
Sep 11 14:16:34.350274 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350176    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-tmp-dir\") pod \"108876d6-417e-45a1-be58-1f7d7109eaba\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") "
Sep 11 14:16:34.350274 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350200    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-model-cache\") pod \"108876d6-417e-45a1-be58-1f7d7109eaba\" (UID: \"108876d6-417e-45a1-be58-1f7d7109eaba\") "
Sep 11 14:16:34.350545 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350487    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-home" (OuterVolumeSpecName: "home") pod "108876d6-417e-45a1-be58-1f7d7109eaba" (UID: "108876d6-417e-45a1-be58-1f7d7109eaba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:16:34.350545 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350530    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-model-cache" (OuterVolumeSpecName: "model-cache") pod "108876d6-417e-45a1-be58-1f7d7109eaba" (UID: "108876d6-417e-45a1-be58-1f7d7109eaba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:16:34.350640 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.350536    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "108876d6-417e-45a1-be58-1f7d7109eaba" (UID: "108876d6-417e-45a1-be58-1f7d7109eaba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:16:34.352632 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.352605    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/108876d6-417e-45a1-be58-1f7d7109eaba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "108876d6-417e-45a1-be58-1f7d7109eaba" (UID: "108876d6-417e-45a1-be58-1f7d7109eaba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:16:34.352926 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.352904    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-dshm" (OuterVolumeSpecName: "dshm") pod "108876d6-417e-45a1-be58-1f7d7109eaba" (UID: "108876d6-417e-45a1-be58-1f7d7109eaba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:16:34.352926 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.352913    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/108876d6-417e-45a1-be58-1f7d7109eaba-kube-api-access-8t6tz" (OuterVolumeSpecName: "kube-api-access-8t6tz") pod "108876d6-417e-45a1-be58-1f7d7109eaba" (UID: "108876d6-417e-45a1-be58-1f7d7109eaba"). InnerVolumeSpecName "kube-api-access-8t6tz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:16:34.442242 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.442209    2563 generic.go:356] "Generic (PLEG): container finished" podID="108876d6-417e-45a1-be58-1f7d7109eaba" containerID="3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f" exitCode=0
Sep 11 14:16:34.442464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.442263    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"
Sep 11 14:16:34.442464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.442298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt" event={"ID":"108876d6-417e-45a1-be58-1f7d7109eaba","Type":"ContainerDied","Data":"3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f"}
Sep 11 14:16:34.442464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.442350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt" event={"ID":"108876d6-417e-45a1-be58-1f7d7109eaba","Type":"ContainerDied","Data":"70213913defe2496c3bb543c6d78892074235c824d562ae07a5b680137c1b55b"}
Sep 11 14:16:34.442464 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.442374    2563 scope.go:117] "RemoveContainer" containerID="3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f"
Sep 11 14:16:34.451047 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.451014    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/108876d6-417e-45a1-be58-1f7d7109eaba-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:16:34.451047 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.451038    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:16:34.451047 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.451047    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t6tz\" (UniqueName: \"kubernetes.io/projected/108876d6-417e-45a1-be58-1f7d7109eaba-kube-api-access-8t6tz\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:16:34.451247 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.451057    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:16:34.451247 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.451066    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:16:34.451247 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.451074    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/108876d6-417e-45a1-be58-1f7d7109eaba-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:16:34.452676 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.452659    2563 scope.go:117] "RemoveContainer" containerID="3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f"
Sep 11 14:16:34.453035 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:16:34.453019    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f\": container with ID starting with 3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f not found: ID does not exist" containerID="3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f"
Sep 11 14:16:34.453098 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.453044    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f"} err="failed to get container status \"3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f\": rpc error: code = NotFound desc = could not find container \"3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f\": container with ID starting with 3ecc72d8a206b5d6b98058ed97aac8fe3703c57e1674764b6abf4e7653e7306f not found: ID does not exist"
Sep 11 14:16:34.466576 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.466545    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"]
Sep 11 14:16:34.469068 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:34.469034    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-e3622e/ipp-route-e3622e-kserve-6cb886bdd7-p8bvt"]
Sep 11 14:16:36.241328 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:16:36.241291    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="108876d6-417e-45a1-be58-1f7d7109eaba" path="/var/lib/kubelet/pods/108876d6-417e-45a1-be58-1f7d7109eaba/volumes"
Sep 11 14:17:13.921191 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:13.921157    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"]
Sep 11 14:17:13.921818 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:13.921392    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx" podUID="fab30bdb-6b4c-404a-a259-6b185a77aa8f" containerName="payload-pre-processing" containerID="cri-o://adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3" gracePeriod=30
Sep 11 14:17:14.181319 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.181251    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:17:14.350330 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.350286    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab30bdb-6b4c-404a-a259-6b185a77aa8f-plugins-config-volume\") pod \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\" (UID: \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\") "
Sep 11 14:17:14.350519 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.350351    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck7vq\" (UniqueName: \"kubernetes.io/projected/fab30bdb-6b4c-404a-a259-6b185a77aa8f-kube-api-access-ck7vq\") pod \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\" (UID: \"fab30bdb-6b4c-404a-a259-6b185a77aa8f\") "
Sep 11 14:17:14.350682 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.350656    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fab30bdb-6b4c-404a-a259-6b185a77aa8f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fab30bdb-6b4c-404a-a259-6b185a77aa8f" (UID: "fab30bdb-6b4c-404a-a259-6b185a77aa8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:17:14.352662 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.352636    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fab30bdb-6b4c-404a-a259-6b185a77aa8f-kube-api-access-ck7vq" (OuterVolumeSpecName: "kube-api-access-ck7vq") pod "fab30bdb-6b4c-404a-a259-6b185a77aa8f" (UID: "fab30bdb-6b4c-404a-a259-6b185a77aa8f"). InnerVolumeSpecName "kube-api-access-ck7vq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:14.451373 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.451281    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab30bdb-6b4c-404a-a259-6b185a77aa8f-plugins-config-volume\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:14.451373 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.451312    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck7vq\" (UniqueName: \"kubernetes.io/projected/fab30bdb-6b4c-404a-a259-6b185a77aa8f-kube-api-access-ck7vq\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:14.689686 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.689654    2563 generic.go:356] "Generic (PLEG): container finished" podID="fab30bdb-6b4c-404a-a259-6b185a77aa8f" containerID="adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3" exitCode=0
Sep 11 14:17:14.689908 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.689711    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"
Sep 11 14:17:14.689908 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.689740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx" event={"ID":"fab30bdb-6b4c-404a-a259-6b185a77aa8f","Type":"ContainerDied","Data":"adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3"}
Sep 11 14:17:14.689908 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.689823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx" event={"ID":"fab30bdb-6b4c-404a-a259-6b185a77aa8f","Type":"ContainerDied","Data":"98755ce2db519fe45cb6e3f09a77a2b07d67b5c766cb6d9ce505a9595075f07f"}
Sep 11 14:17:14.689908 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.689845    2563 scope.go:117] "RemoveContainer" containerID="adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3"
Sep 11 14:17:14.700127 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.700107    2563 scope.go:117] "RemoveContainer" containerID="adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3"
Sep 11 14:17:14.700427 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:17:14.700407    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3\": container with ID starting with adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3 not found: ID does not exist" containerID="adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3"
Sep 11 14:17:14.700499 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.700437    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3"} err="failed to get container status \"adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3\": rpc error: code = NotFound desc = could not find container \"adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3\": container with ID starting with adc4e5c37d2ad4a9b52f85edaa616b122e0c9e722724c6392ceb0721731eecd3 not found: ID does not exist"
Sep 11 14:17:14.712816 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.712724    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"]
Sep 11 14:17:14.716463 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:14.716437    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-68d0c7-7d774ccc6b-q2pvx"]
Sep 11 14:17:16.241457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:16.241413    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fab30bdb-6b4c-404a-a259-6b185a77aa8f" path="/var/lib/kubelet/pods/fab30bdb-6b4c-404a-a259-6b185a77aa8f/volumes"
Sep 11 14:17:17.235990 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:17.235958    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:17:20.545024 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:20.544985    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"]
Sep 11 14:17:20.545470 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:20.545288    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy" containerID="cri-o://4544c7a5b896d0c8d5c23e8465d7a85f6438386e0b70961a02da9e3cd5f15084" gracePeriod=30
Sep 11 14:17:25.761531 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.761491    2563 generic.go:356] "Generic (PLEG): container finished" podID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerID="4544c7a5b896d0c8d5c23e8465d7a85f6438386e0b70961a02da9e3cd5f15084" exitCode=0
Sep 11 14:17:25.762017 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.761570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" event={"ID":"eccc2735-0e68-4a09-9d6b-5668c5544790","Type":"ContainerDied","Data":"4544c7a5b896d0c8d5c23e8465d7a85f6438386e0b70961a02da9e3cd5f15084"}
Sep 11 14:17:25.762017 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.761654    2563 scope.go:117] "RemoveContainer" containerID="e5d28542dd97265bab117b767f5d6de74db6ed7540584022bf238e6a665597f4"
Sep 11 14:17:25.818451 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.818384    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:17:25.853784 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.853739    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-podinfo\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.853990 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.853836    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-socket\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.853990 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.853960    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-credential-socket\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.854121 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854049    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-envoy\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.854121 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854078    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlflz\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-kube-api-access-mlflz\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.854121 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854112    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-token\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.854287 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854140    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:25.854287 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854149    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-certs\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.854287 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854168    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:25.854287 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854281    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-data\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.854508 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854321    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccc2735-0e68-4a09-9d6b-5668c5544790-istiod-ca-cert\") pod \"eccc2735-0e68-4a09-9d6b-5668c5544790\" (UID: \"eccc2735-0e68-4a09-9d6b-5668c5544790\") "
Sep 11 14:17:25.854508 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854427    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:25.854642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854619    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-data" (OuterVolumeSpecName: "istio-data") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:25.854976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854924    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:25.854976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854949    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:25.854976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854955    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eccc2735-0e68-4a09-9d6b-5668c5544790-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:17:25.854976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854966    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:25.855233 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.854982    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:25.856215 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.856187    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:17:25.856597 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.856573    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-token" (OuterVolumeSpecName: "istio-token") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:25.856804 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.856780    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:25.856878 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.856798    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-kube-api-access-mlflz" (OuterVolumeSpecName: "kube-api-access-mlflz") pod "eccc2735-0e68-4a09-9d6b-5668c5544790" (UID: "eccc2735-0e68-4a09-9d6b-5668c5544790"). InnerVolumeSpecName "kube-api-access-mlflz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:25.956436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.956386    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccc2735-0e68-4a09-9d6b-5668c5544790-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:25.956436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.956430    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:25.956436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.956439    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:25.956436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.956448    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlflz\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-kube-api-access-mlflz\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:25.956436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:25.956458    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccc2735-0e68-4a09-9d6b-5668c5544790-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:26.768671 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:26.768634    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"
Sep 11 14:17:26.769233 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:26.768677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m" event={"ID":"eccc2735-0e68-4a09-9d6b-5668c5544790","Type":"ContainerDied","Data":"b5d17361219a75adb166c0c8d6f0c00d74446cc9f0018946b8fa01f1885c426f"}
Sep 11 14:17:26.769233 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:26.768736    2563 scope.go:117] "RemoveContainer" containerID="4544c7a5b896d0c8d5c23e8465d7a85f6438386e0b70961a02da9e3cd5f15084"
Sep 11 14:17:26.788094 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:26.788061    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"]
Sep 11 14:17:26.791871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:26.791839    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-68d0c7-openshift-default-7c4ddcfd4-t746m"]
Sep 11 14:17:28.241379 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:28.241342    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" path="/var/lib/kubelet/pods/eccc2735-0e68-4a09-9d6b-5668c5544790/volumes"
Sep 11 14:17:31.466412 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:31.466368    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"]
Sep 11 14:17:31.468847 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:31.468824    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-932c97-29818935-d6d2r"]
Sep 11 14:17:32.240856 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:32.240826    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0a13829-a586-4b96-98b1-e96b6dbeab85" path="/var/lib/kubelet/pods/d0a13829-a586-4b96-98b1-e96b6dbeab85/volumes"
Sep 11 14:17:34.531426 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:34.531388    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:17:38.533561 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:38.533525    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"]
Sep 11 14:17:38.534036 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:38.533847    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy" containerID="cri-o://8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849" gracePeriod=30
Sep 11 14:17:38.994119 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:38.994085    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:17:38.994303 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:38.994156    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:17:43.791929 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.791898    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:17:43.875688 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.875657    2563 generic.go:356] "Generic (PLEG): container finished" podID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerID="8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849" exitCode=0
Sep 11 14:17:43.875893 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.875712    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"
Sep 11 14:17:43.875893 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.875741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" event={"ID":"d724c8eb-7e5b-4ce7-8860-b01f4d037837","Type":"ContainerDied","Data":"8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849"}
Sep 11 14:17:43.875893 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.875810    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng" event={"ID":"d724c8eb-7e5b-4ce7-8860-b01f4d037837","Type":"ContainerDied","Data":"fb6bdcd1acc79d56d5e02ec5213b809ef81555b9cfcec05d266ee65963390bca"}
Sep 11 14:17:43.875893 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.875835    2563 scope.go:117] "RemoveContainer" containerID="8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849"
Sep 11 14:17:43.885315 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.885295    2563 scope.go:117] "RemoveContainer" containerID="3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2"
Sep 11 14:17:43.894786 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.894763    2563 scope.go:117] "RemoveContainer" containerID="8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849"
Sep 11 14:17:43.895080 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:17:43.895063    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849\": container with ID starting with 8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849 not found: ID does not exist" containerID="8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849"
Sep 11 14:17:43.895199 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.895091    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849"} err="failed to get container status \"8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849\": rpc error: code = NotFound desc = could not find container \"8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849\": container with ID starting with 8d41e15247439c0bdc35e0794ac7dc8202f1174933538daeb3c50436f2f81849 not found: ID does not exist"
Sep 11 14:17:43.895199 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.895109    2563 scope.go:117] "RemoveContainer" containerID="3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2"
Sep 11 14:17:43.895403 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:17:43.895388    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2\": container with ID starting with 3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2 not found: ID does not exist" containerID="3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2"
Sep 11 14:17:43.895453 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.895407    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2"} err="failed to get container status \"3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2\": rpc error: code = NotFound desc = could not find container \"3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2\": container with ID starting with 3f72ade7e01a7d394b775e8f0d4071d75b60592cde5494f4df217617567ceed2 not found: ID does not exist"
Sep 11 14:17:43.930807 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.930765    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-podinfo\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.930988 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.930885    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-certs\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.930988 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.930916    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-credential-socket\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.930988 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.930940    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-data\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.930988 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.930987    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-token\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.931209 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931028    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-envoy\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.931209 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgqqz\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-kube-api-access-kgqqz\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.931209 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931083    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istiod-ca-cert\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.931209 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931108    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-socket\") pod \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\" (UID: \"d724c8eb-7e5b-4ce7-8860-b01f4d037837\") "
Sep 11 14:17:43.931421 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931321    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:43.931421 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931336    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:43.931421 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931353    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-data" (OuterVolumeSpecName: "istio-data") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:43.931576 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931533    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:17:43.931773 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931588    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:43.931947 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931926    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:43.932092 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931971    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:43.932092 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.931987    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:43.932092 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.932000    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:43.932092 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.932012    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:43.933329 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.933303    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:17:43.933436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.933415    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:43.933825 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.933805    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-token" (OuterVolumeSpecName: "istio-token") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:43.933886 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:43.933835    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-kube-api-access-kgqqz" (OuterVolumeSpecName: "kube-api-access-kgqqz") pod "d724c8eb-7e5b-4ce7-8860-b01f4d037837" (UID: "d724c8eb-7e5b-4ce7-8860-b01f4d037837"). InnerVolumeSpecName "kube-api-access-kgqqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:44.033386 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:44.033347    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:44.033386 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:44.033379    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:44.033386 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:44.033390    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d724c8eb-7e5b-4ce7-8860-b01f4d037837-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:44.033645 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:44.033400    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgqqz\" (UniqueName: \"kubernetes.io/projected/d724c8eb-7e5b-4ce7-8860-b01f4d037837-kube-api-access-kgqqz\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:17:44.200266 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:44.200231    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"]
Sep 11 14:17:44.205305 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:44.205275    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-932c97-openshift-default-5bc74d985d-8ccng"]
Sep 11 14:17:44.240942 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:44.240909    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" path="/var/lib/kubelet/pods/d724c8eb-7e5b-4ce7-8860-b01f4d037837/volumes"
Sep 11 14:17:52.331737 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:52.331702    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:17:56.398641 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:56.398607    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"]
Sep 11 14:17:56.399106 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:17:56.398869    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy" containerID="cri-o://c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5" gracePeriod=30
Sep 11 14:18:01.443704 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.443659    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:18:01.656201 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.656168    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:18:01.712871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.712738    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-certs\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.712871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.712829    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc7a8bb-403d-4865-bf9d-94959070730b-istiod-ca-cert\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.712871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.712865    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-socket\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.713161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.712891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-data\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.713161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.712919    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-credential-socket\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.713161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713033    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:18:01.713161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-podinfo\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.713161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713099    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-token\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.713161 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713149    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-envoy\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.713430 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713163    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:18:01.713430 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713192    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz2zs\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-kube-api-access-sz2zs\") pod \"1cc7a8bb-403d-4865-bf9d-94959070730b\" (UID: \"1cc7a8bb-403d-4865-bf9d-94959070730b\") "
Sep 11 14:18:01.713430 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713232    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:18:01.713430 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713250    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:18:01.713430 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713322    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cc7a8bb-403d-4865-bf9d-94959070730b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:18:01.713685 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713534    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.713685 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713554    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc7a8bb-403d-4865-bf9d-94959070730b-istiod-ca-cert\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.713685 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713567    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-workload-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.713685 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713581    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-data\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.713685 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.713594    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-credential-socket\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.715491 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.715460    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-kube-api-access-sz2zs" (OuterVolumeSpecName: "kube-api-access-sz2zs") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "kube-api-access-sz2zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:18:01.715610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.715489    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:18:01.715610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.715542    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:18:01.715821 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.715795    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cc7a8bb-403d-4865-bf9d-94959070730b" (UID: "1cc7a8bb-403d-4865-bf9d-94959070730b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:18:01.814690 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.814650    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-podinfo\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.814690 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.814680    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-token\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.814690 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.814693    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc7a8bb-403d-4865-bf9d-94959070730b-istio-envoy\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.814976 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.814704    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz2zs\" (UniqueName: \"kubernetes.io/projected/1cc7a8bb-403d-4865-bf9d-94959070730b-kube-api-access-sz2zs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:18:01.980610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.980521    2563 generic.go:356] "Generic (PLEG): container finished" podID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerID="c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5" exitCode=0
Sep 11 14:18:01.980610 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.980571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" event={"ID":"1cc7a8bb-403d-4865-bf9d-94959070730b","Type":"ContainerDied","Data":"c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5"}
Sep 11 14:18:01.980829 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.980612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s" event={"ID":"1cc7a8bb-403d-4865-bf9d-94959070730b","Type":"ContainerDied","Data":"12b7cf21b32839c73580dfa6b38215ab7f423524d35ecf8fb8fa534cccd67d06"}
Sep 11 14:18:01.980829 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.980634    2563 scope.go:117] "RemoveContainer" containerID="c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5"
Sep 11 14:18:01.980829 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.980575    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"
Sep 11 14:18:01.990637 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.990615    2563 scope.go:117] "RemoveContainer" containerID="cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27"
Sep 11 14:18:02.000094 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:01.999967    2563 scope.go:117] "RemoveContainer" containerID="c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5"
Sep 11 14:18:02.000399 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:18:02.000373    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5\": container with ID starting with c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5 not found: ID does not exist" containerID="c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5"
Sep 11 14:18:02.000529 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:02.000412    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5"} err="failed to get container status \"c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5\": rpc error: code = NotFound desc = could not find container \"c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5\": container with ID starting with c0631fd3ddb109fa54afe7ada0b8cb9922ef92e1be366ce4ef9a31a0c1d811d5 not found: ID does not exist"
Sep 11 14:18:02.000529 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:02.000440    2563 scope.go:117] "RemoveContainer" containerID="cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27"
Sep 11 14:18:02.000820 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:18:02.000795    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27\": container with ID starting with cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27 not found: ID does not exist" containerID="cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27"
Sep 11 14:18:02.000905 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:02.000830    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27"} err="failed to get container status \"cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27\": rpc error: code = NotFound desc = could not find container \"cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27\": container with ID starting with cd5052d87dd93ef5433a17988ae094fd3ded431ec678f853e2f3cbbe3a418e27 not found: ID does not exist"
Sep 11 14:18:02.002872 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:02.002850    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"]
Sep 11 14:18:02.007565 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:02.007542    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-367517-openshift-default-5b6b88d848-b867s"]
Sep 11 14:18:02.241653 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:02.241573    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" path="/var/lib/kubelet/pods/1cc7a8bb-403d-4865-bf9d-94959070730b/volumes"
Sep 11 14:18:24.440937 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:24.440905    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:18:24.447795 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:24.447770    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:18:24.686820 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:24.686779    2563 scope.go:117] "RemoveContainer" containerID="6104e3962f6c9cc0fa9e8b6b237ff1462b3be9810bed5b77eecdb03260ca390b"
Sep 11 14:18:24.701268 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:18:24.701243    2563 scope.go:117] "RemoveContainer" containerID="6883cb501da6e277f76f700ef772c046f368c01070cdb2948e83d1ce74944986"
Sep 11 14:19:22.931787 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:19:22.931728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:19:50.949511 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:19:50.949424    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:19:59.435231 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:19:59.435191    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:20:08.436534 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:20:08.436492    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:20:16.229495 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:20:16.229458    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:20:29.040312 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:20:29.040271    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:20:35.226441 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:20:35.226408    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:21:24.756770 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:21:24.756672    2563 scope.go:117] "RemoveContainer" containerID="a41214c7d588c140bbfd8ccb09486e1dfb8e076475fbbf8f987b8605aeb297d3"
Sep 11 14:21:24.768689 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:21:24.768661    2563 scope.go:117] "RemoveContainer" containerID="d9535a97647c7da7360865ad3d6cce8eaf55d3a10aaac36881dc8e132e8e22b9"
Sep 11 14:22:02.146712 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:02.146673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:22:08.970782 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:08.970728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:22:17.954642 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:17.954598    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:22:24.794700 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:24.794665    2563 scope.go:117] "RemoveContainer" containerID="7df8a43e236638626fa54f248c3d5067ae83448f6aacc0c3c9ba20b3a8ce0be3"
Sep 11 14:22:25.942855 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:25.942811    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:22:27.168457 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168418    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"]
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168799    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168812    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168821    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="108876d6-417e-45a1-be58-1f7d7109eaba" containerName="main"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168826    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="108876d6-417e-45a1-be58-1f7d7109eaba" containerName="main"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168834    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168840    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168849    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fab30bdb-6b4c-404a-a259-6b185a77aa8f" containerName="payload-pre-processing"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168855    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fab30bdb-6b4c-404a-a259-6b185a77aa8f" containerName="payload-pre-processing"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168862    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619ba29d-0766-4f2a-8694-d31e3ccc78a0" containerName="revoke-keys"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168867    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="619ba29d-0766-4f2a-8694-d31e3ccc78a0" containerName="revoke-keys"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168877    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168883    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168894    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy"
Sep 11 14:22:27.168891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168900    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168910    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168915    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168969    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168977    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="619ba29d-0766-4f2a-8694-d31e3ccc78a0" containerName="revoke-keys"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168982    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168987    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="108876d6-417e-45a1-be58-1f7d7109eaba" containerName="main"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168993    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.168998    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc7a8bb-403d-4865-bf9d-94959070730b" containerName="istio-proxy"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.169006    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy"
Sep 11 14:22:27.169331 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.169011    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fab30bdb-6b4c-404a-a259-6b185a77aa8f" containerName="payload-pre-processing"
Sep 11 14:22:27.171957 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.171940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.174828 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.174738    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-5327b004-kserve-self-signed-certs\""
Sep 11 14:22:27.183341 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.183313    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"]
Sep 11 14:22:27.248160 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.248123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-home\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.248339 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.248163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tmp-dir\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.248339 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.248187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-model-cache\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.248339 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.248236    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k85lv\" (UniqueName: \"kubernetes.io/projected/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-kube-api-access-k85lv\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.248339 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.248272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tls-certs\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.248484 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.248341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-dshm\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.349910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.349873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-dshm\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.350141 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.349990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-home\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.350141 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.350021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tmp-dir\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.350141 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.350063    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-model-cache\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.350141 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.350104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k85lv\" (UniqueName: \"kubernetes.io/projected/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-kube-api-access-k85lv\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.350380 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.350172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tls-certs\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.350465 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.350439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-home\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.350578 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.350557    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-model-cache\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.350643 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.350620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tmp-dir\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.352483 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.352449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-dshm\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.352724 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.352709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tls-certs\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.359037 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.359014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k85lv\" (UniqueName: \"kubernetes.io/projected/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-kube-api-access-k85lv\") pod \"e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") " pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.481978 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.481886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:27.758665 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:27.758637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"]
Sep 11 14:22:27.760691 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:22:27.760661    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb81a48c_a648_4a57_b8cc_2b6d6d935d9d.slice/crio-0b4ea95f1dba6d9e8d0c476fea501c1f7297a2dc7dfc7694c4baf7af7dad3b71 WatchSource:0}: Error finding container 0b4ea95f1dba6d9e8d0c476fea501c1f7297a2dc7dfc7694c4baf7af7dad3b71: Status 404 returned error can't find the container with id 0b4ea95f1dba6d9e8d0c476fea501c1f7297a2dc7dfc7694c4baf7af7dad3b71
Sep 11 14:22:28.492242 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:28.492211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr" event={"ID":"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d","Type":"ContainerStarted","Data":"969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e"}
Sep 11 14:22:28.492242 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:28.492255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr" event={"ID":"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d","Type":"ContainerStarted","Data":"0b4ea95f1dba6d9e8d0c476fea501c1f7297a2dc7dfc7694c4baf7af7dad3b71"}
Sep 11 14:22:28.511273 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:28.511213    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr" podStartSLOduration=1.511191083 podStartE2EDuration="1.511191083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:22:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:22:28.510714129 +0000 UTC m=+2044.768914141" watchObservedRunningTime="2026-09-11 14:22:28.511191083 +0000 UTC m=+2044.769391093"
Sep 11 14:22:35.673245 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:35.673203    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:22:37.482924 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:37.482873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:37.483453 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:37.482956    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:37.496127 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:37.496093    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:37.552370 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:37.552336    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:22:51.879871 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:51.879829    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"]
Sep 11 14:22:52.566862 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:52.566824    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:22:59.781954 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:22:59.781914    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:23:05.672217 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:05.672178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:23:06.903914 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:06.903871    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr" podUID="bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" containerName="main" containerID="cri-o://969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e" gracePeriod=45
Sep 11 14:23:07.159468 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.159402    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:23:07.225547 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-home\") pod \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") "
Sep 11 14:23:07.225726 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225557    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-model-cache\") pod \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") "
Sep 11 14:23:07.225726 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225584    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k85lv\" (UniqueName: \"kubernetes.io/projected/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-kube-api-access-k85lv\") pod \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") "
Sep 11 14:23:07.225726 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225598    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-dshm\") pod \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") "
Sep 11 14:23:07.225957 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tmp-dir\") pod \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") "
Sep 11 14:23:07.225957 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225887    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tls-certs\") pod \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\" (UID: \"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d\") "
Sep 11 14:23:07.225957 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225888    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-home" (OuterVolumeSpecName: "home") pod "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" (UID: "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:23:07.225957 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225904    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" (UID: "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:23:07.226158 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.225989    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" (UID: "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:23:07.226263 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.226247    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-home\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:23:07.226320 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.226268    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-model-cache\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:23:07.226320 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.226283    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tmp-dir\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:23:07.228008 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.227982    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-dshm" (OuterVolumeSpecName: "dshm") pod "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" (UID: "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:23:07.228117 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.228017    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-kube-api-access-k85lv" (OuterVolumeSpecName: "kube-api-access-k85lv") pod "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" (UID: "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d"). InnerVolumeSpecName "kube-api-access-k85lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:23:07.228117 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.228040    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" (UID: "bb81a48c-a648-4a57-b8cc-2b6d6d935d9d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:23:07.326939 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.326897    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k85lv\" (UniqueName: \"kubernetes.io/projected/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-kube-api-access-k85lv\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:23:07.327122 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.327036    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-dshm\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:23:07.327122 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.327060    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d-tls-certs\") on node \"ip-10-0-137-192.ec2.internal\" DevicePath \"\""
Sep 11 14:23:07.710922 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.710889    2563 generic.go:356] "Generic (PLEG): container finished" podID="bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" containerID="969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e" exitCode=0
Sep 11 14:23:07.711147 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.710932    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"
Sep 11 14:23:07.711147 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.710978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr" event={"ID":"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d","Type":"ContainerDied","Data":"969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e"}
Sep 11 14:23:07.711147 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.711024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr" event={"ID":"bb81a48c-a648-4a57-b8cc-2b6d6d935d9d","Type":"ContainerDied","Data":"0b4ea95f1dba6d9e8d0c476fea501c1f7297a2dc7dfc7694c4baf7af7dad3b71"}
Sep 11 14:23:07.711147 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.711045    2563 scope.go:117] "RemoveContainer" containerID="969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e"
Sep 11 14:23:07.721534 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.721510    2563 scope.go:117] "RemoveContainer" containerID="969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e"
Sep 11 14:23:07.721909 ip-10-0-137-192 kubenswrapper[2563]: E0911 14:23:07.721885    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e\": container with ID starting with 969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e not found: ID does not exist" containerID="969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e"
Sep 11 14:23:07.721992 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.721921    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e"} err="failed to get container status \"969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e\": rpc error: code = NotFound desc = could not find container \"969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e\": container with ID starting with 969535da8b2b47b7dce74f60bb942bdfc6ccc320f39457321e338d7842e8dc5e not found: ID does not exist"
Sep 11 14:23:07.735701 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.735667    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"]
Sep 11 14:23:07.740087 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:07.740054    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-5327b004-kserve-69978f978c-wlpxr"]
Sep 11 14:23:08.240659 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:08.240621    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" path="/var/lib/kubelet/pods/bb81a48c-a648-4a57-b8cc-2b6d6d935d9d/volumes"
Sep 11 14:23:12.947673 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:12.947625    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:23:20.956425 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:20.956381    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:23:24.479778 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:24.479731    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:23:24.492181 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:24.492155    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:23:27.352008 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:27.351965    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:23:33.362320 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:33.362282    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:23:40.706653 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:40.706603    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-wgs8h"]
Sep 11 14:23:49.640745 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:49.640710    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b5cnq_65cc6a5b-677d-4947-a203-45b4fe333db4/manager/0.log"
Sep 11 14:23:49.921383 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:49.921298    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6l2mf_9fb5273f-bd3e-4614-9789-2d7f2209d5e1/server/0.log"
Sep 11 14:23:50.069135 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:50.069097    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cgrfd_0a7c8a77-7e73-464e-b4d4-e82523801d0e/manager/0.log"
Sep 11 14:23:50.200325 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:50.200243    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85cf544d9c-2bc9x_d765dc6a-5737-4b56-9f63-7f7c51efd209/copy-manifests/0.log"
Sep 11 14:23:50.213142 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:50.213106    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85cf544d9c-2bc9x_d765dc6a-5737-4b56-9f63-7f7c51efd209/manager/0.log"
Sep 11 14:23:52.301435 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:52.301395    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-df96r_bf708184-4e4f-426c-9115-4e230f40b1fe/manager/0.log"
Sep 11 14:23:52.554396 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:52.554320    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-vrxch_6cc99932-beba-4190-a0fd-8748cb766402/kuadrant-console-plugin/0.log"
Sep 11 14:23:52.826698 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:52.826620    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-wgs8h_6a409f61-fb88-4581-a8d8-937b7a304931/limitador/0.log"
Sep 11 14:23:53.320493 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:53.320453    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm_2b6449a4-e6ad-4bf8-b322-c6b995583ee8/istio-proxy/0.log"
Sep 11 14:23:53.824400 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:53.824371    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c2426_8253d448-24ad-4ad7-9cf6-3ec341ef5bdf/istio-proxy/1.log"
Sep 11 14:23:54.220193 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:54.220164    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-565b697964-gmnzh_d967ed16-c979-45a0-aef2-4c4ccf73c12e/router/0.log"
Sep 11 14:23:55.219365 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:55.219334    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-szh5n_07242df5-b0d4-4049-bd30-630fe493b02d/main/0.log"
Sep 11 14:23:55.340213 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:23:55.340188    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-4strr_fcf4d975-a308-48b9-a8b4-e73c3c8a2a36/main/0.log"
Sep 11 14:24:00.874904 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.874859    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kdmhs/must-gather-lb2ns"]
Sep 11 14:24:00.876241 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.876211    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy"
Sep 11 14:24:00.876423 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.876411    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="eccc2735-0e68-4a09-9d6b-5668c5544790" containerName="istio-proxy"
Sep 11 14:24:00.876529 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.876518    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" containerName="main"
Sep 11 14:24:00.876600 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.876592    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" containerName="main"
Sep 11 14:24:00.876910 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.876894    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb81a48c-a648-4a57-b8cc-2b6d6d935d9d" containerName="main"
Sep 11 14:24:00.877038 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.877027    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d724c8eb-7e5b-4ce7-8860-b01f4d037837" containerName="istio-proxy"
Sep 11 14:24:00.881450 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.881422    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdmhs/must-gather-lb2ns"
Sep 11 14:24:00.885375 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.885345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kdmhs\"/\"default-dockercfg-fw2z6\""
Sep 11 14:24:00.885559 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.885529    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kdmhs\"/\"kube-root-ca.crt\""
Sep 11 14:24:00.885559 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.885542    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kdmhs\"/\"openshift-service-ca.crt\""
Sep 11 14:24:00.912678 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:00.912632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdmhs/must-gather-lb2ns"]
Sep 11 14:24:01.016452 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.016421    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gppw\" (UniqueName: \"kubernetes.io/projected/bd533c3b-d96c-4c0f-b692-31578604b6e5-kube-api-access-4gppw\") pod \"must-gather-lb2ns\" (UID: \"bd533c3b-d96c-4c0f-b692-31578604b6e5\") " pod="openshift-must-gather-kdmhs/must-gather-lb2ns"
Sep 11 14:24:01.016640 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.016460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bd533c3b-d96c-4c0f-b692-31578604b6e5-must-gather-output\") pod \"must-gather-lb2ns\" (UID: \"bd533c3b-d96c-4c0f-b692-31578604b6e5\") " pod="openshift-must-gather-kdmhs/must-gather-lb2ns"
Sep 11 14:24:01.117195 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.117157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gppw\" (UniqueName: \"kubernetes.io/projected/bd533c3b-d96c-4c0f-b692-31578604b6e5-kube-api-access-4gppw\") pod \"must-gather-lb2ns\" (UID: \"bd533c3b-d96c-4c0f-b692-31578604b6e5\") " pod="openshift-must-gather-kdmhs/must-gather-lb2ns"
Sep 11 14:24:01.117195 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.117200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bd533c3b-d96c-4c0f-b692-31578604b6e5-must-gather-output\") pod \"must-gather-lb2ns\" (UID: \"bd533c3b-d96c-4c0f-b692-31578604b6e5\") " pod="openshift-must-gather-kdmhs/must-gather-lb2ns"
Sep 11 14:24:01.117570 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.117554    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bd533c3b-d96c-4c0f-b692-31578604b6e5-must-gather-output\") pod \"must-gather-lb2ns\" (UID: \"bd533c3b-d96c-4c0f-b692-31578604b6e5\") " pod="openshift-must-gather-kdmhs/must-gather-lb2ns"
Sep 11 14:24:01.129316 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.129237    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gppw\" (UniqueName: \"kubernetes.io/projected/bd533c3b-d96c-4c0f-b692-31578604b6e5-kube-api-access-4gppw\") pod \"must-gather-lb2ns\" (UID: \"bd533c3b-d96c-4c0f-b692-31578604b6e5\") " pod="openshift-must-gather-kdmhs/must-gather-lb2ns"
Sep 11 14:24:01.191557 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.191518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdmhs/must-gather-lb2ns"
Sep 11 14:24:01.474727 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.474703    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdmhs/must-gather-lb2ns"]
Sep 11 14:24:01.476899 ip-10-0-137-192 kubenswrapper[2563]: W0911 14:24:01.476865    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd533c3b_d96c_4c0f_b692_31578604b6e5.slice/crio-06e602847cac5782796a0f3efc17d073cc6d8d6611da251bf65be5de0797d1b7 WatchSource:0}: Error finding container 06e602847cac5782796a0f3efc17d073cc6d8d6611da251bf65be5de0797d1b7: Status 404 returned error can't find the container with id 06e602847cac5782796a0f3efc17d073cc6d8d6611da251bf65be5de0797d1b7
Sep 11 14:24:01.478843 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:01.478826    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:24:02.019378 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:02.019346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdmhs/must-gather-lb2ns" event={"ID":"bd533c3b-d96c-4c0f-b692-31578604b6e5","Type":"ContainerStarted","Data":"06e602847cac5782796a0f3efc17d073cc6d8d6611da251bf65be5de0797d1b7"}
Sep 11 14:24:03.029895 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:03.028974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdmhs/must-gather-lb2ns" event={"ID":"bd533c3b-d96c-4c0f-b692-31578604b6e5","Type":"ContainerStarted","Data":"09f90853533912d5d7b854b5946575e2a06ec9b3a7639728a4f223904ca74f18"}
Sep 11 14:24:03.029895 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:03.029033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdmhs/must-gather-lb2ns" event={"ID":"bd533c3b-d96c-4c0f-b692-31578604b6e5","Type":"ContainerStarted","Data":"0b261c09a0f2e310a2d40539310f1a83577a1fc0a70ac2aa03a63ead2891687a"}
Sep 11 14:24:03.060179 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:03.060122    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kdmhs/must-gather-lb2ns" podStartSLOduration=2.397006625 podStartE2EDuration="3.060104099s" totalImagesPullingTime="663.097474ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:24:03.058122835 +0000 UTC m=+2139.316322834" watchObservedRunningTime="2026-09-11 14:24:03.060104099 +0000 UTC m=+2139.318304107"
Sep 11 14:24:04.026948 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:04.026913    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dg5cq_5fa3726c-9b76-4680-8d8b-0931a221c76f/global-pull-secret-syncer/0.log"
Sep 11 14:24:04.454707 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:04.454665    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x4cs8_db06779b-7d1e-4c26-b85c-1346bb5a1a01/konnectivity-agent/0.log"
Sep 11 14:24:04.552676 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:04.552637    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-192.ec2.internal_f63e32dd172790f2b9a6ee29b1c6f139/haproxy/0.log"
Sep 11 14:24:09.619833 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:09.619792    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-df96r_bf708184-4e4f-426c-9115-4e230f40b1fe/manager/0.log"
Sep 11 14:24:09.736221 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:09.736190    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-vrxch_6cc99932-beba-4190-a0fd-8748cb766402/kuadrant-console-plugin/0.log"
Sep 11 14:24:09.870066 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:09.869976    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-wgs8h_6a409f61-fb88-4581-a8d8-937b7a304931/limitador/0.log"
Sep 11 14:24:12.006196 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:12.006149    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-hrqh9_b373479a-c8a5-4ba1-9a2f-918bf37b97cb/cluster-monitoring-operator/0.log"
Sep 11 14:24:12.524794 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:12.524743    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r89ms_f085f235-e492-4ef6-a75c-e07f6faa8397/node-exporter/0.log"
Sep 11 14:24:12.581618 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:12.581585    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r89ms_f085f235-e492-4ef6-a75c-e07f6faa8397/kube-rbac-proxy/0.log"
Sep 11 14:24:12.615353 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:12.615324    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r89ms_f085f235-e492-4ef6-a75c-e07f6faa8397/init-textfile/0.log"
Sep 11 14:24:13.959376 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:13.959335    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"]
Sep 11 14:24:13.964698 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:13.964672    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:13.983437 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:13.983399    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"]
Sep 11 14:24:14.059516 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.059160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdwx5\" (UniqueName: \"kubernetes.io/projected/7f1eb656-3875-4ae3-8375-5f55639d9661-kube-api-access-wdwx5\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.059516 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.059231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-lib-modules\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.059516 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.059285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-podres\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.059516 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.059335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-proc\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.059516 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.059400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-sys\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.160593 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdwx5\" (UniqueName: \"kubernetes.io/projected/7f1eb656-3875-4ae3-8375-5f55639d9661-kube-api-access-wdwx5\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.160796 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-lib-modules\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.160796 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160640    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-podres\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.160796 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-proc\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.160796 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160791    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-proc\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.161053 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-sys\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.161053 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160878    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-lib-modules\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.161053 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-sys\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.161053 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.160949    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7f1eb656-3875-4ae3-8375-5f55639d9661-podres\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.171949 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.171917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdwx5\" (UniqueName: \"kubernetes.io/projected/7f1eb656-3875-4ae3-8375-5f55639d9661-kube-api-access-wdwx5\") pod \"perf-node-gather-daemonset-rgtbd\" (UID: \"7f1eb656-3875-4ae3-8375-5f55639d9661\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.279065 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.278982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:14.598103 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.598009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"]
Sep 11 14:24:14.813330 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:14.813294    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-rw8gf_4b4a3162-901d-416a-8e12-569888476830/networking-console-plugin/0.log"
Sep 11 14:24:15.132064 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:15.131576    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd" event={"ID":"7f1eb656-3875-4ae3-8375-5f55639d9661","Type":"ContainerStarted","Data":"b7e37bfd43d5bea7a9913335a2a1d3688b85aa571114259dd84a8c09212822bf"}
Sep 11 14:24:15.132064 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:15.131644    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd" event={"ID":"7f1eb656-3875-4ae3-8375-5f55639d9661","Type":"ContainerStarted","Data":"fecf9bb8a8f5fbf261f002c1c25680313300f9f6a52cd6391d3ba6461c4675f2"}
Sep 11 14:24:15.132064 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:15.131865    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:15.158462 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:15.158405    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd" podStartSLOduration=2.158381821 podStartE2EDuration="2.158381821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 14:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:24:15.156194766 +0000 UTC m=+2151.414394775" watchObservedRunningTime="2026-09-11 14:24:15.158381821 +0000 UTC m=+2151.416581827"
Sep 11 14:24:15.465891 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:15.465814    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/1.log"
Sep 11 14:24:15.472862 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:15.472825    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wvb64_e274ee6d-0a73-411d-b62a-e41022c1be0a/console-operator/2.log"
Sep 11 14:24:16.031986 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:16.031959    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-74rmp_e819c83e-d5a1-43ba-b752-25ec41500a36/download-server/0.log"
Sep 11 14:24:16.651554 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:16.651520    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-hps9n_b7f01e20-525e-49ae-98fd-4ca29745b391/volume-data-source-validator/0.log"
Sep 11 14:24:17.837994 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:17.837966    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hkkx6_4b97da6d-72f4-4fa8-ae6f-6f8d2f325550/dns/0.log"
Sep 11 14:24:17.877317 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:17.877284    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hkkx6_4b97da6d-72f4-4fa8-ae6f-6f8d2f325550/kube-rbac-proxy/0.log"
Sep 11 14:24:18.053243 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:18.053212    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2tdnj_78099fbe-6524-4477-8da7-1184017e0ed8/dns-node-resolver/0.log"
Sep 11 14:24:18.842814 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:18.842786    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nns9f_2adefd72-0d9f-40aa-a193-6b68449ce681/node-ca/0.log"
Sep 11 14:24:19.906417 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:19.906390    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm_2b6449a4-e6ad-4bf8-b322-c6b995583ee8/istio-proxy/0.log"
Sep 11 14:24:20.231804 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:20.231696    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c2426_8253d448-24ad-4ad7-9cf6-3ec341ef5bdf/istio-proxy/0.log"
Sep 11 14:24:20.241780 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:20.241720    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c2426_8253d448-24ad-4ad7-9cf6-3ec341ef5bdf/istio-proxy/1.log"
Sep 11 14:24:20.350929 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:20.350903    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-565b697964-gmnzh_d967ed16-c979-45a0-aef2-4c4ccf73c12e/router/0.log"
Sep 11 14:24:21.153595 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:21.153563    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-rgtbd"
Sep 11 14:24:21.291073 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:21.291046    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7x4q2_2bf035ca-5e6b-47ee-9bb2-9f66aa68c6f3/serve-healthcheck-canary/0.log"
Sep 11 14:24:21.882569 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:21.882524    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-w7d2b_59c08c9a-f359-4fa7-b3a2-4b04500dd53e/insights-operator/0.log"
Sep 11 14:24:21.883178 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:21.883152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-w7d2b_59c08c9a-f359-4fa7-b3a2-4b04500dd53e/insights-operator/1.log"
Sep 11 14:24:22.129478 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:22.129441    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xdn8q_89f217f3-bc97-424a-9084-27bf60ffcc6a/kube-rbac-proxy/0.log"
Sep 11 14:24:22.158541 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:22.158449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xdn8q_89f217f3-bc97-424a-9084-27bf60ffcc6a/exporter/0.log"
Sep 11 14:24:22.189043 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:22.189011    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xdn8q_89f217f3-bc97-424a-9084-27bf60ffcc6a/extractor/0.log"
Sep 11 14:24:24.972147 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:24.972110    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b5cnq_65cc6a5b-677d-4947-a203-45b4fe333db4/manager/0.log"
Sep 11 14:24:25.113052 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:25.112981    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6l2mf_9fb5273f-bd3e-4614-9789-2d7f2209d5e1/server/0.log"
Sep 11 14:24:25.182551 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:25.182519    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm_2b6449a4-e6ad-4bf8-b322-c6b995583ee8/istio-proxy/0.log"
Sep 11 14:24:25.238903 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:25.238818    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cgrfd_0a7c8a77-7e73-464e-b4d4-e82523801d0e/manager/0.log"
Sep 11 14:24:25.292514 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:25.292483    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85cf544d9c-2bc9x_d765dc6a-5737-4b56-9f63-7f7c51efd209/manager/0.log"
Sep 11 14:24:25.361991 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:25.361959    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85cf544d9c-2bc9x_d765dc6a-5737-4b56-9f63-7f7c51efd209/copy-manifests/0.log"
Sep 11 14:24:25.519051 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:25.518972    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c2426_8253d448-24ad-4ad7-9cf6-3ec341ef5bdf/istio-proxy/0.log"
Sep 11 14:24:25.528084 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:25.528049    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c2426_8253d448-24ad-4ad7-9cf6-3ec341ef5bdf/istio-proxy/1.log"
Sep 11 14:24:25.640286 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:25.640252    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-565b697964-gmnzh_d967ed16-c979-45a0-aef2-4c4ccf73c12e/router/0.log"
Sep 11 14:24:27.182013 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:27.181965    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8546dd6968-rvcwb_624e3791-d936-477d-b457-d1bd485a16d1/manager/0.log"
Sep 11 14:24:27.282436 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:27.282388    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4vcpm_2b6449a4-e6ad-4bf8-b322-c6b995583ee8/istio-proxy/0.log"
Sep 11 14:24:27.613377 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:27.613345    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c2426_8253d448-24ad-4ad7-9cf6-3ec341ef5bdf/istio-proxy/0.log"
Sep 11 14:24:27.623047 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:27.623015    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-c2426_8253d448-24ad-4ad7-9cf6-3ec341ef5bdf/istio-proxy/1.log"
Sep 11 14:24:27.731552 ip-10-0-137-192 kubenswrapper[2563]: I0911 14:24:27.731515    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-565b697964-gmnzh_d967ed16-c979-45a0-aef2-4c4ccf73c12e/router/0.log"